0.7.2-13 : Added SCAMLWriter and fixed hard breaks to be more like th…

…e actual Markdown spec.

literate/01-Introduction.markdown

@@ -25,13 +25,31 @@ See the [Recipes](10-Usage/02-Recipes.html) page for more.
 
 
 
+## Possible Updates
+
+- Move the 'knockoff.latex' package to the 'extra' package.
+
+- Investigate SSP pass-through, it might have to be extended on top of HTML
+
+
+
 ## Recent Updates ##
 
+
 ### `0.7.2-SNAPSHOT`
 
-**TODO** Move the 'knockoff.latex' package to the 'extra' package.
+* Added a SCAML writer
+
+
+### `0.7.1-12` June 1, 2010
+
+Two small bug fixes:
+
+1. A reference link followed by a paren was being matched as a normal link.
 
-**TODO** Investigate SSP pass-through, it might have to be extended on top of HTML
+1. If you used an asterix-delimited em right after a list item asterix marker, the
+   line is now considered to be a list item if you use an odd number of asterixes
+   on the line. (Ugly, but generally going to be OK.)
 
 
 ### `0.7.1-11` May 17, 2010

literate/30-Implementation/10-Parsing_Markdown/12-Chunk_Types.markdown

@@ -21,7 +21,8 @@ grabbed the spanning elements of each block, to construct the final `Block` mode
 
 Here is where I can apply hard breaks in the middle of paragraphs. If we've
 recognized a `Text` span that contains two spaces and a newline, we split the span
-sequence at this point into two lists, and then append two blocks.
+sequence at this point into two lists, and then append two blocks. One of them will
+be an `HTMLSpan(<br/>)`.
 
     // The TextChunk
     case class TextChunk( val content : String ) extends Chunk {
@@ -31,25 +32,30 @@ sequence at this point into two lists, and then append two blocks.
                           spans : Seq[Span], position : Position,
                           discounter : Discounter ) {
 
-        val (lead, rest) = splitAtHardBreak( spans, new ListBuffer )
-        list += Paragraph( lead, position )
-        if ( ! rest.isEmpty )
-          appendNewBlock( list, remaining, rest, position, discounter )
+        val split = splitAtHardBreak( spans, new ListBuffer )
+        list += Paragraph( split, position )
       }
 
       def splitAtHardBreak( spans : Seq[Span], cur : Buffer[Span] )
-                          : (Seq[Span], Seq[Span]) = {
-        if ( spans.isEmpty ) return ( cur, spans )
+                          : Seq[Span] = {
+        if ( spans.isEmpty ) return cur
         spans.first match {
           case text : Text =>
-            text.content.indexOf("  \n") match {
+            // Skip past whitespace in the case we have some HTML.
+            var start = 0
+            if ( ! cur.isEmpty && cur.last.isInstanceOf[HTMLSpan] )
+              while ( start < text.content.length &&
+                      Character.isWhitespace( text.content(start) ) )
+                start = start + 1
+            text.content.indexOf("  \n", start) match {
               case -1 => {}
               case idx =>
                 val end = idx + "  \n".length
-                val (c1, c2) = ( text.content.substring( 0, end ),
+                val (c1, c2) = ( text.content.substring( 0, idx ),
                                  text.content.substring( end ) )
                 cur += Text(c1)
-                return ( cur, List(Text(c2)) ++ spans.drop(1) )
+                cur += HTMLSpan("<br/>\n")
+                return splitAtHardBreak( List(Text(c2)) ++ spans.drop(1), cur )
             }
           case _ => {}
         }

literate/30-Implementation/50-Extra/04-SCAML.markdown

@@ -27,7 +27,7 @@ once the document is converted.
 
     // The SCAMLDiscounter trait
     trait SCAMLDiscounter extends Discounter with StringExtras
-      with SCAMLXHTMLWriter with SCAMLPlainTextWriter {
+      with SCAMLXHTMLWriter with SCAMLPlainTextWriter with SCAMLWriter {
       
       override def knockoff( source : CharSequence ) : Seq[Block] =
         super.knockoff( source ).map( convertSCAMLBlock )
@@ -98,14 +98,204 @@ once the document is converted.
 
 ## Writing ##
 
-Generally speaking, SCAML is just going to be passed through without much fuss or
-assistance of any kind (since we're assuming that the sequences will be dealt with
-by a templating engine). We want to avoid any XML escapes, etc. So each of the
-writers just needs to be adjusted to handle the `InterpolatedSCAML` type.
+SCAML is a pretty different structure, in that the indentation level is pretty
+significant to documents.
+
+Note that the only SCAML that should be possibly present in the markdown document
+is the short interpolated kind, which looks like this:
+
+    In the last product version #{lastProduct} we introduced...
+
+Because of the nature of matching operators used right now, we need to add latex and
+plain text writers.
+
+
+    // The SCAML Writer
+    trait SCAMLWriter { self : TextWriter =>
+      
+      // TODO -> theIndent and currentIndent actually shouldn't be accessible. 
+      val theIndent = "  "
+      
+      val currentIndent = new StringBuilder
+      
+      def indent = currentIndent.toString
+      
+      def increaseIndent = currentIndent.append( theIndent )
+      
+      def decreaseIndent = currentIndent.delete( currentIndent.length - theIndent.length,
+                                                 currentIndent.length )
+      
+      
+      /** Generates a SCAML version of the markdown document. */
+      def toSCAML( blocks : Seq[Block] ) : String = {
+        implicit val writer = new StringWriter
+        blocksToSCAML( blocks )
+        writer.toString
+      }
+      
+      def blocksToSCAML( blocks : Seq[Block] )( implicit writer : Writer ) : Unit =
+        blocks.foreach( blockToSCAML )
+      
+      /** Each block level element starts a new line, with indent, and increases
+          the current indent level before continuing with what it contains.
+          When done, it reduces the indent level. */
+      def blockToSCAML( block : Block )( implicit writer : Writer ) : Unit = {
+        if ( block.isInstanceOf[LinkDefinition] ) return
+        writer.write( indent )
+        block match {          
+          case Paragraph( _, _ ) => writer.write( "%p\n" )
+          case Header( level, _, _ ) =>
+            writer.write( "%h" )
+            writer.write( level.toString )
+            writer.write( "\n" )
+          case Blockquote( _, _ ) =>
+            writer.write( "%blockquote\n" )
+          case CodeBlock( _, _ ) => 
+            writer.write( "%pre\n" )
+            increaseIndent
+            writer.write( indent )
+            writer.write( "%code\n" )
+          case HorizontalRule( _ ) =>
+            writer.write( "%hr\n" )
+          case OrderedItem( _, _ ) | UnorderedItem( _, _ ) =>
+            writer.write( "%li\n" )
+          case OrderedList( _ ) =>
+            writer.write( "%ol\n" )
+          case UnorderedList( _ ) =>
+            writer.write( "%ul\n" )
+          case _ => error( "Unknown block: " + block )
+        }
+        increaseIndent
+        block match {
+          case Paragraph( spans, _ ) => spans.foreach( spanToSCAML )
+          case Header( _, spans, _ ) => spans.foreach( spanToSCAML )
+          case Blockquote( children, _ ) => children.foreach( blockToSCAML )
+          case CodeBlock( text, _ ) =>
+            // **TODO** Prefix all lines with the indent
+            val indented = text.content.split("\n")
+                                       .map{ l => indent + l }
+                                       .mkString("\n")
+            writer.write( indented + "\n" )
+            decreaseIndent
+          case HorizontalRule( _ ) => {}
+          case OrderedItem( children, _ ) => children.foreach( blockToSCAML )
+          case UnorderedItem( children, _ ) => children.foreach( blockToSCAML )
+          case OrderedList( items ) => items.foreach( blockToSCAML )
+          case UnorderedList( items ) => items.foreach( blockToSCAML )
+          case _ => error( "Illegal block: " + block )
+        }
+        decreaseIndent
+      }
+      
+      /** Delegate to the appropriate handler to decide on indentation. */
+      def spanToSCAML( span : Span )( implicit writer : Writer ) : Unit = {
+        span match {
+          case Text(_) | HTMLSpan(_) | CodeSpan(_) => normalSpanToSCAML( span )
+          case Strong(_) | Emphasis(_) | Link(_,_,_) | IndirectLink(_,_) |
+               ImageLink(_,_,_) | IndirectImageLink(_,_) => spanBlockToSCAML( span )
+        }
+      }
+      
+      /** These spans operate like the block level elements above. */
+      def spanBlockToSCAML( span : Span )( implicit writer : Writer ) : Unit = {
+        writer.write( indent )
+        span match {
+          case Strong( _ ) => writer.write( "%strong\n")
+          case Emphasis( children ) => writer.write( "%em\n" )
+          case Link( _, url, title ) =>
+            writer.write( "%a{ :href => \"" )
+            writer.write( url )
+            writer.write( "\"" )
+            for ( t <- title ) {
+              writer.write( ", :title => \"" )
+              writer.write( t.replace("\"", "\\\"").replace("\n", " ").trim )
+              writer.write( "\"" )
+            }
+            writer.write( " }\n" )
+          case IndirectLink( _, definition ) =>
+            writer.write( "%a{ :href => \"" )
+            writer.write( definition.url )
+            writer.write( "\"" )
+            for ( t <- definition.title ) {
+              writer.write( ", :title => \"" )
+              writer.write( t.replace("\"", "\\\"").replace("\n", " ").trim )
+              writer.write( "\"" )
+            }
+            writer.write( " }\n" )
+          case ImageLink( children, url, title ) =>
+            writer.write( "%img{ :src => \"" )
+            writer.write( url )
+            writer.write( "\"" )
+            for ( t <- title ) {
+              writer.write( ", :title => \"" )
+              writer.write( t.replace("\"", "\\\"").replace("\n", " ").trim )
+              writer.write( "\"" )
+            }
+            if ( ! children.isEmpty ) {
+              val stringWriter = new StringWriter
+              children.foreach{ c => spanToText(c)(stringWriter) }
+              val escaped = stringWriter.toString.replace( "\"", "\\\"" ).replace("\n", " ").trim
+              writer.write(", :alt => \"" )
+              writer.write( escaped )
+              writer.write( "\"" )
+            }
+            writer.write( " }\n" )
+          case IndirectImageLink( children, definition ) =>
+            writer.write( "%img{ :src => \"" )
+            writer.write( definition.url )
+            writer.write( "\"" )
+            for ( t <- definition.title ) {
+              writer.write( ", :title => \"" )
+              writer.write( t.replace("\"", "\\\"") )
+              writer.write( "\"" )
+            }
+            if ( ! children.isEmpty ) {
+              val stringWriter = new StringWriter
+              children.foreach{ c => spanToText(c)(stringWriter) }
+              val escaped = stringWriter.toString.replace( "\"", "\\\"" ).replace("\n", " ").trim
+              writer.write(", :alt => \"" )
+              writer.write( escaped )
+              writer.write( "\"" )
+            }
+            writer.write( " }\n" )          
+          case _ => error( "Illegal block span: " + span )
+        }
+        increaseIndent
+        span match {
+          case Strong( children ) => children.foreach( spanToSCAML )
+          case Emphasis( children ) => children.foreach( spanToSCAML )
+          case Link( children, _, _ ) => children.foreach( spanToSCAML )
+          case IndirectLink( children, _ ) => children.foreach( spanToSCAML )
+          case ImageLink( _, _, _ ) => {}
+          case IndirectImageLink( _, _ ) => {}
+          case _ => error( "Illegal block span: " + span )
+        }
+        decreaseIndent
+      }
 
-    // The SCAML writers
-    trait SCAMLXHTMLWriter extends XHTMLWriter {
       
+      /** We generally take each span and put it on it's own line. */
+      def normalSpanToSCAML( span : Span )( implicit writer : Writer ) : Unit = {
+        writer.write( indent )
+        span match {
+          case Text( content ) => writer.write( content )
+          case HTMLSpan( html ) => writer.write( html )
+          case CodeSpan( code ) =>
+            writer.write( "%code " )
+            writer.write( code.replace("\n", " ") ) // There shouldn't be newlines
+                                                    // in here anyway
+          case _ => error( "Illegal normal span: " + span )
+        }
+        writer.write( "\n" )
+      }
+    }
+
+If you're "feeling lucky" you can attempt to use the XHTML version of the document
+with just a few interpolated strings, but you need to watch out for any line that
+might appear to be a SCAML statement. This is not a good general solution.
+
+    // The SCAML XHTML Writer
+    trait SCAMLXHTMLWriter extends XHTMLWriter {
       override def spanToXHTML : Span => Node = span => {
         span match {
           case InterpolatedSCAML( content ) => Unparsed( content )
@@ -114,17 +304,17 @@ writers just needs to be adjusted to handle the `InterpolatedSCAML` type.
       }
     }
 
+    // The SCAML Latex Writer
     trait SCAMLLatexWriter extends LatexWriter {
-      
       override def toLatex( span : Span )( implicit writer : Writer ) : Unit =
         span match {
           case InterpolatedSCAML( content ) => writer.write( content )
           case _ => super.toLatex( span )
         }
     }
 
+    // The SCAML Plain Text Writer
     trait SCAMLPlainTextWriter extends TextWriter {
-     
       override def spanToText( span : Span )( implicit writer : Writer ) : Unit = {
         span match {
           case InterpolatedSCAML( content ) => writer.write( content + " " )
@@ -138,14 +328,20 @@ writers just needs to be adjusted to handle the `InterpolatedSCAML` type.
 
     import com.tristanhunt.knockoff._
     import com.tristanhunt.knockoff.latex.{ LatexWriter }
-    import java.io.Writer
+    import java.io.{ Writer, StringWriter }
     import scala.xml.{ Node, Unparsed }
 
     // See the InterpolatedSCAML class
 
     // See the SCAMLDiscounter trait
 
-    // See the SCAML writers
+    // See the SCAML Writer
+
+    // See the SCAML XHTML Writer
+
+    // See the SCAML Latex Writer
+
+    // See the SCAML Plain Text Writer
 
 
     // In test com/tristanhunt/knockoff/extra/SCAMLSpec.scala

literate/30-Implementation/50-Integration_Testing.markdown

@@ -75,6 +75,17 @@ This will indicate which direction we use for comparison.
             fromText should equal ( to.text )
           }
         }
+        
+        it( "should convert tests from Markdown to SCAML" ) {
+          val dir = file( basedir, "wholesaler_markdown-scaml" )
+          dir.listTests(".txt", ".scaml").foreach { case (from, to) =>
+            from should be ('exists)
+            to should be ('exists)
+            println( "Test: " + from.getName )
+            val fromSCAML = toSCAML( knockoff( from.text ) ).trim
+            fromSCAML should equal( to.text )
+          }
+        }
       }
     }
 

notes/0.7.2-13.markdown

@@ -0,0 +1,6 @@
+* Added the `SCAMLWriter` with associated `toSCAML` method, in case you want to
+  pepper your markdown files with SCAML - and then use [Scalate](http://scalate.fusesource.org)
+  to process them. (_Note_ No SSP... yet.)
+
+* Specification bugfix whereupon hard breaks are properly broken with `<br/>`s
+  instead of splitting the two lines into separate paragraphs.

project/build.properties

@@ -3,7 +3,7 @@
 project.name=knockoff
 project.organization=com.tristanhunt
 sbt.version=0.7.4
-project.version=0.7.1-12
+project.version=0.7.2-13
 def.scala.version=2.7.7
 build.scala.versions=2.8.0.RC3 2.8.0.RC2 2.8.0.RC1 2.8.0.Beta1 2.7.7 2.7.6 2.7.5 2.7.4
 project.initialize=false