Markdown functions

Here is an example of a package used to produce HTML files from a MarkDown document used in this document project.

This example is about creating HTML documents from a MarkDown source. The MarkDown document format is a prefered format for technical documentation since its focus is on the text, code examples that can be copied and pasted into the reader's source code, images and links are easy to implement. In the ACF language, we have included functions to convert MarkDown to HTML.

We have chosen to create the ACF manual in this way, as we want to be able to have an HTML folder to include the distribution package, containing all the documents necessary for the manual - i.e. no server technology needed for the manual.

The process includes a small FileMaker application, - a table with preferences where all the Path's and fixed stuff goes. A document table for the manual documents, and a category table for the categories of the manual pages. We have a template for new documents that make it easy to start new ones.

I use LightPaper for editing - so the ".md" extension is bound to LightPaper. When I launch a document from the FileMaker Application, it opens in LightPaper Automatically.

We have a folder for the destination HTML documents, and another for the source .md documents. Inside the HTML folder, we have put the themes folder that comes with the plugin, a CSS folder for extra CSS, and a images folder for images used in the manual.

When converting Markdown documents, there are three phases.

  1. Pre-processing: We have included images in the MarkDown documents. They can reside anywhere - typically from some server volume or directly from the editors desktop. These images need to be copied into the HTML/images folder so that they are accessible from the web-pages. Then the image URL has to be re-written in the MarkDown document to point into its new location. The image URL in the MarkDown document has to be relative from the document location; thus we need to have a prefix on it that later has to be removed from the resulting HTML.
  2. The Markdown to HTML conversion
  3. Post processing - add left column menu and header to the document.

For point 1: Here is the different Paths we need to take care of:


Original Image path when dragged into the MarkDown document from the Desktop:  
../../../Desktop/imageab.png

Relative Image path for its new location as seen from the MarkDown document: 
 ../html_root_folder/images/imageab.png

Resulting image path in the HTML document: 
images/imageab.png

The absolute path of the document folder is: 
/Users/ole/Projects/ACFmanual/mdDocs/

The absolute path of the HTML path is: 
/Users/ole/Projects/ACFmanual/html_root_folder/

Now, we have some filesystem fun to pick apart all these paths and copy files. This is done in the MoveImages function seen below.

Example Listing:

The Package Header:


package MarkDownFunctions "MarkDown funksjoner for dok prosjektet..."; 

The function for preprocessing the MarkDown document and copy image files:

The purpose of the next function below is to extract all the image tags from the Markdown document, rewrite the image URL, copy the files into the desired location for the HTML generation in the next step. The Image tag has this form:


![image 1] (../../../Desktop/regex-explanation.png)

We use this regex to extract all occurrences of such an image tag; even we don't know the URL or anything more than the format.


string regex = "!\[[^\]]*\]\(((.+?\/)?([^\/)]+))\)";

This is explained this way: (Screen shot from regex101.com)

image 1

Here is the function:


function MoveImages ( string SourceFile, string html_root, string relpath ) 

    print "MoveImages  enters...."; 
    int x = open ( SourceFile, "r"  ); 
    string md = read (x ) ; 
    close ( x ) ; 
    print "Pre prosessing"; 
    string regex = "!\[[^\]]*\]\(((.+?\/)?([^\/)]+))\)"; // used to extract image tags...
    // gr 0 = full match, gr 1 = full path, gr 2 = directory path, gr 3 = file path. 
    array string tags = regex_extract ( regex, md ) ; 
    int no = sizeof ( tags ) ; 
    if ( no == 0 ) then
        print "Pre prosessing - no tags"; 
        return "";      
    end if
    
// Getting the Markdown document folder where relative images starts from. 
    string docFolder = regex_replace ( "^(.+\/)[^\/]+$", SourceFile, "\1" ) ; 

    int i, cnt = 0; 
    string fulltag, fullpath, fileFullPath, dir, file, newPath, newRelPath, newtag, res; 
    
// Loop all tags, and process each one of them. 
    for ( i= 1, no)
        print "\n" + tags[i]; 
        fulltag = getValue ( tags[i], 1);
        fullpath = getValue ( tags[i], 2);
        fileFullPath = fullpath; 
        dir = getValue ( tags[i], 3);
        file = getValue ( tags[i], 4);
        if ( file_exists ( fullpath ) == 0 ) then
             // probably a relative path. Prefix with doc folder to get absolute path. 
            fullpath = docFolder + fullpath; 
        end if
        if ( file_exists ( fullpath ) ) then
            newRelPath = relpath + "images/" + file; 
            newPath = html_root + "/images/" + file; 
            if ( file_exists ( newPath ) == 0) then
                res = copy_file ( fullpath, newPath ) ; 
                if ( res == "OK") then
                    newtag = substitute ( fulltag, fileFullPath, newRelPath ) ; 
                    md = substitute ( md, fulltag, newtag ) ; 
                    cnt++; 
                end if
            end if
        end if
    end for
    if ( cnt > 0 ) then
        // save the modified MarkDown document. Take a backup first, in case we did something nasty...
        res = copy_file ( SourceFile, SourceFile+".bak" ) ; 
        // Write back to file. 
        x = open ( SourceFile, "w"  ); 
        write ( x, md ) ; 
        close ( x ) ;
    end if
    return "OK"; 
END

The Post processing function to add header and left column to the document:


function post_processing (string htmlfile, string removeText)

/* Called from ConvertMarkdown 
    Open the generated HTML document, that is a plain page of the markdown doc, 
    and put on a header, and a left navigation bar.
   */

    print "post processingn";
    int x = open ( htmlfile, "r"  ); 
    string html = read (x ) ; 
    close ( x ) ; 
    string body_part = between ( html, "<body>", "</body>" ) ; 
    string head_part = between ( html, "<head>", "</head>" ) ; 
    string stylesheet = '<link rel="stylesheet" type="text/css" href="include/css/style.css">';
    string top_bar = Preferences::topp_nav; 
    
    // Build the left nav
    string left_nav = Preferences::Left_nav; 
     
doclist = @ExecuteSQL ( "SELECT Title, slug_filename, Category FROM ACF_Documents LEFT OUTER JOIN ACF_Categories as c ON c.PrimaryKey = Category_UUID ORDER BY Sorting, Category" ; "||" ; "|*|")@;
    
    print doclist;  // Debugg print to consolle. 

    array string docs = explode ("|*|", doclist ), flt ; 
    int noDocs = sizeof ( docs ); 
    int i; 
    string curCat = "noone"; 
    for (i = 1, noDocs)
        flt = explode ( "||", docs[i]); 
        if (flt [3] != curCat ) then
            if ( curCat != "noone") then
                left_nav += "</ul>\n"; 
            end if
            left_nav += "<h4>" + flt[3] + "</h4>\n"; 
            left_nav += '<ul class="linklist">\n'; 
            curCat = flt[3]; 
        end if
        left_nav += "<li>" + flt[1] + "</li>\n"; 
    end for
    left_nav += "</ul>\n"; 
    
    // Merge the parts
    array string fileparts = {"<!DOCTYPE html>\n<head>", head_part, stylesheet, "</head>\n<body>", 
        '<div class="hd">', top_bar, 
        '</div><div class="content_band"><div class="left">',left_nav, 
        '</div><div class="content">', body_part,  
        "</div></div>", "</body>n</html>\n" }; 
        
    string result = implode ( "\n", fileparts ) ; 

    // Remove Image path prefixes 
    result = substitute (result, removeText, ""); 
    
    // substitute all titles in the documents with a link to that page. 
    for (i = 1, noDocs)
        flt = explode ( "||", docs[i]); 
        result = substitute (result, flt[1], '<a href="'+flt[2]+'">' + flt[1]+"</a>"); 
    end for

    // Write back to file. 
    x = open ( htmlfile, "w"  ); 
    write ( x, result ) ; 
    close ( x ) ;
    print "End-post-processingn"; 
    return "OK"; 
end

The markdown document conversion function:

This function below uses the two functions above to complete the full conversion.


function ConvertMarkdown (string sourcefile, string htmlfile, string html_root, string style, string codestyle, string removeText)
    
    print "Convert markdown enters....\n"; 
    set_markdown_html_root ( html_root ) ; 
    string sf = sourcefile, df = htmlfile; 
    $$SourceFile = ""; 
    if (sf == "") then
        sf = select_file ("Velg en Markdown Fil?"); 
        $$SourceFile = sf; 
    end if
    if (sf != "") then
        if (df == "") then
         df = save_file_dialogue ("Lagre fil som?", "xx.html", html_root);
        end if
        if (df != "") then
                 // Pre processing - Handle images in document. 
            string res = MoveImages ( sf, html_root, removeText );
                // Main conversion
            res = markdown2html (sf, style+","+codestyle, df); 
                // Post processing...
            res = post_processing (df, removeText); 
        end if 
    end if
    return "OK";
end

A small utility function to produce html filenames:


function createHTML_Filename ( string file, string html_path ) 

    string newfile = substitute ( lower(file), " ", "_"); 
    newfile = substitute ( newfile, ":", "_"); 

    if (right(html_path, 1) != "/") then
        html_path += "/"; 
    end if
    return html_path + newfile + ".html";
end