diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 0000000..e69de29 diff --git a/htt/examples/implementing-types-in-go/index.html b/htt/examples/implementing-types-in-go/index.html new file mode 100644 index 0000000..b99a5fb --- /dev/null +++ b/htt/examples/implementing-types-in-go/index.html @@ -0,0 +1,793 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

Implementing types in Go

+

+ This example implements some of the AST nodes from the book, Writing an Interpreter in Go, specifically as implemented in this git repository.

+ +

+ You will see nodes generally come in two types, Statement and Expression, implementing the empty method of either to mark the type of node. + Beyond this, nodes implement String and TokenLiteral, and sometimes these are implemented in the same way.

+ + +

Getting Started

+

+ We start by creating a template file, ast.htt and defining the package, implementing the base types and so on. All of this is regular Go code:

+ +
+
+
+
% @component main
+package ast;
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/zanshin/interpreter/token"
+)
+
+type Node interface {
+	TokenLiteral() string
+	String() string
+}
+
+type Statement interface {
+	Node
+	statementNode()
+}
+
+type Expression interface {
+	Node
+	expressionNode()
+}
+% @end
+
+
+
+ + Rendering this essentially produces the same output: +
+
+
+
package ast;
+            
+            import (
+            	"bytes"
+            	"strings"
+            
+            	"github.com/zanshin/interpreter/token"
+            )
+            
+            type Node interface {
+            	TokenLiteral() string
+            	String() string
+            }
+            
+            type Statement interface {
+            	Node
+            	statementNode()
+            }
+            
+            type Expression interface {
+            	Node
+            	expressionNode()
+            }
+
+
+
+ +

Rendering Nodes

+

+ Now let's start rendering nodes.

+ +

Defining the model

+

+ To start, let's define some data describing the nodes we want. When speaking of code generation, the data we are generating code from is often called the model. For larger projects, I would store the model in a separate file. That way, template files could refer to it.

+ +

+ For small, self-contained projects, we can embed the model with the template. When compiling a template file (ast.htt), HTT will look for a corresponding Lua file (ast.htt.lua) matching the name of the template file, with the prefix .lua.

+ +

+ Hence, here is the ast.htt.lua file to accompany our ast.htt template file: +

+
+
+
function nodes()
+    return {
+        {
+			is_expr = true,
+			short = "i",
+			name = "Identifier",
+			fields = {
+				{ name = "Value", type = "string" }
+			}
+		},
+        {
+			is_expr = true,
+			short = "il",
+			name = "IntegerLiteral",
+			fields = {
+				{ name = "Value", type = "int64" }
+			}
+		},
+    }
+end
+
+
+

+ +

+ The code in ast.htt.lua is, as the name implies, regular Lua code. The function nodes returns a list (table) of (associative) tables, each describing a type of AST node. I like to wrap the data in a function to ensure I cannot modify global state and impact other rendering code.

+ +

Defining how to render a node

+

+ To use this data, let's define a component, node, and just render out the name of the component for now:

+
+
+
+
% @component node
+{{ctx.name}}
+% @end
+
+
+
+ +

Defining how to render all nodes

+ To render all nodes, we define a component render_nodes, which loops over each entry in the model (the nodes function in ast.htt.lua) and renders it: + +
+
+
+
% @component render_nodes
+% for _, elem in ipairs(nodes()) do
+{{@ node elem }}
+% end
+% @end
+
+
+
+ +

+ You could add these lines directly to main. I normally would. But this way I can show you changes to this component in isolation.

+ +

Putting it all together

+ +

+ Finally, we call render_nodes from our main component. Since we have no data to pass to render_nodes, we pass an empty Lua table ({}). + + Our main component, thus becomes: +

+
+
+
+
% @component main
+package ast;
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/zanshin/interpreter/token"
+)
+
+type Node interface {
+	TokenLiteral() string
+	String() string
+}
+
+type Statement interface {
+	Node
+	statementNode()
+}
+
+type Expression interface {
+	Node
+	expressionNode()
+}
+
+{{@ render_nodes {} }}
+% @end
+
+
+
+ +

Output

+
+
+
+
package ast;
+            
+            import (
+            	"bytes"
+            	"strings"
+            
+            	"github.com/zanshin/interpreter/token"
+            )
+            
+            type Node interface {
+            	TokenLiteral() string
+            	String() string
+            }
+            
+            type Statement interface {
+            	Node
+            	statementNode()
+            }
+            
+            type Expression interface {
+            	Node
+            	expressionNode()
+            }
+            
+            Identifier
+            IntegerLiteral
+
+
+
+ + Success! We see the names of the nodes we defined in our model below the Go code we started out with. Now we can start defining in earnest how the AST nodes are to be rendered. + +

Rendering the Node struct

+ Let's start by rendering the struct for east AST node: + +
+
+
+
% @component node
+type {{ctx.name}} struct {
+    Token token.Token
+    % for _, field in ipairs(ctx.fields) do
+    {{field.name}} {{field.type}}
+    % end
+}
+% @end
+
+
+
+ + The nodes now render as: +
+
+
+
type Identifier struct {
+                Token token.Token
+                Value string
+            }
+            type IntegerLiteral struct {
+                Token token.Token
+                Value int64
+            }
+
+
+
+ + Before moving on, let's add some whitespace between components. You can do this in multiple ways, but inserting an empty line ahead of rendering each component in loop body of render_nodes works well: +
+
+
+
% @component render_nodes
+% for _, elem in ipairs(nodes()) do
+
+{{@ node elem }}
+% end
+% @end
+
+
+
+ + Now we get: +
+
+
+

+            type Identifier struct {
+                Token token.Token
+                Value string
+            }
+            
+            type IntegerLiteral struct {
+                Token token.Token
+                Value int64
+            }
+
+
+
+ +

Implementing the node interface

+ For now, we only handle nodes whose String and TokenLiteral implementations both return the value of .Token.Literal. + Beyond that, we must implement either the empty function expressionNode() or statementNode(), depending on the type of node, as identified by the is_expr field for the node in our model: + +
+
+
+
% @component node
+type {{ctx.name}} struct {
+    Token token.Token
+    % for _, field in ipairs(ctx.fields) do
+    {{field.name}} {{field.type}}
+    % end
+}
+
+% if ctx.is_expr == true then
+func ({{ctx.short}} *{{ctx.name}}) expressionNode() {}
+% else
+func ({{ctx.short}} *{{ctx.name}}) statementNode() {}
+% end
+
+func ({{ctx.short}} *{{ctx.name}}) TokenLiteral() string {
+  return {{ctx.short}}.Token.Literal
+}
+
+func ({{ctx.short}} *{{ctx.name}}) String() string {
+  return {{ctx.short}}.Token.Literal
+}
+% @end
+
+
+
+ + Now, the code for the nodes themselves becomes: +
+
+
+

+            type Identifier struct {
+                Token token.Token
+                Value string
+            }
+            
+            func (i *Identifier) expressionNode() {}
+            
+            func (i *Identifier) TokenLiteral() string {
+              return i.Token.Literal
+            }
+            
+            func (i *Identifier) String() string {
+              return i.Token.Literal
+            }
+            
+            type IntegerLiteral struct {
+                Token token.Token
+                Value int64
+            }
+            
+            func (il *IntegerLiteral) expressionNode() {}
+            
+            func (il *IntegerLiteral) TokenLiteral() string {
+              return il.Token.Literal
+            }
+            
+            func (il *IntegerLiteral) String() string {
+              return il.Token.Literal
+            }
+
+
+
+ +

Overriding the String method

+

+ Some nodes implement a custom String() method to stringify themselves and their contents. + To support this, let us extend the model such that each node can define a string_fn attribute, pointing to a component which implements the String() method.

+ +

Referencing components from the model.

+

+ Recall that we defined our components in the template ast.htt and put our accompanying model in ast.htt.lua, and that it is these two files combined which will

+ +

+ To support this, we need a way to reference the components to use for implementing the String() method. One way to solve this is to send along a reference to the module generated from the template file itself.

+ +

+ All templates can use the variable M to refer to their own modules. So we can just send this along to the nodes() function which returns the model:

+ +
+
+
+
% @component render_nodes
+% for _, elem in ipairs(nodes(M)) do
+
+{{@ node elem }}
+% end
+% @end
+
+
+
+ + +

+ From the model function, we can now refer to components in the template by their name. So m.node would refer to the node component defined in ast.htt

+ +
+
+
+
function nodes(m)
+    -- `m` is a reference to the template module, now we can reference
+    -- components like `node` as `m.node`.
+end
+
+
+
+ +

+ With this change, we update the model, expanding the number of nodes we implement. Some, like the ReturnStatement node, will now use a custom component (here: return_statement_string) to implement the body of their String() method.

+
+
+
+
function nodes(m)
+	return {
+		{
+			is_expr = true,
+			short = "i",
+			name = "Identifier",
+			fields = {
+				{ name = "Value", type = "string" }
+			}
+		},
+		{
+			is_expr = false,
+			short = "rs",
+			name = "ReturnStatement",
+			fields = {
+				{ name = "ReturnValue", type = "Expression" }
+			},
+			string_fn = m.return_statement_string,
+		},
+		{
+			is_expr = false,
+			short = "es",
+			name = "ExpressionStatement",
+			fields = {
+				{ name = "Expression", type = "Expression" }
+			},
+			string_fn = m.expr_statement_string,
+		},
+		{
+			is_expr = true,
+			short = "il",
+			name = "IntegerLiteral",
+			fields = {
+				{ name = "Value", type = "int64" }
+			}
+		},
+		{
+			is_expr = true,
+			short = "pe",
+			name = "PrefixExpression",
+			fields = {
+				{ name = "Operator", type = "string" },
+				{ name = "Right",    type = "Expression" },
+			},
+			string_fn = m.prefix_expr_string,
+		}
+	}
+end
+
+
+
+
+ +

Composing components

+

+ The key change in the template is how we implement node Notice now that we check if ctx.string_fn is defined, and if so, renders that component:

+ +
+
+
+
% @component node
+type {{ctx.name}} struct {
+    Token token.Token
+    % for _, field in ipairs(ctx.fields) do
+    {{field.name}} {{field.type}}
+    % end
+}
+
+% if ctx.is_expr == true then
+func ({{ctx.short}} *{{ctx.name}}) expressionNode() {}
+% else
+func ({{ctx.short}} *{{ctx.name}}) statementNode() {}
+% end
+
+func ({{ctx.short}} *{{ctx.name}}) TokenLiteral() string {
+  return {{ctx.short}}.Token.Literal
+}
+
+func ({{ctx.short}} *{{ctx.name}}) String() string {
+  % if ctx.string_fn then
+  {{@ ctx.string_fn ctx }}
+  % else
+  return {{ctx.short}}.Token.Literal
+  % end
+}
+% @end
+
+
+
+ +

+ This demonstrates how HTT components can be passed as arguments to other components which can render them. Also notice that since all compontents take exactly one table argument, we can pass all arguments along by passing ctx. + These two facts combine to make components very composable.

+ +

Implementing the custom String() methods

+

+ The final change to the ast.htt template file is implementing the components for the custom String() methods.

+ +

+ There is nothing to these components aside from noting that since we passed the entire ctx from the node component along to these, we still have access to attributes like short from the model describing the node.

+ +
+
+
+
% @component return_statement_string
+var out bytes.Buffer
+
+out.WriteString(rs.Token.Literal + " ")
+if rs.ReturnValue != nil {
+	out.WriteString(rs.ReturnValue.String())
+}
+
+out.WriteString(";")
+
+return out.String()
+% @end
+
+
+
+ +
+
+
+
% @component expr_statement_string
+if {{ctx.short}}.Expression != nil {
+	return {{ctx.short}}.Expression.String()
+}
+return ""
+% @end
+
+
+
+ +
+
+
+
% @component prefix_expr_string
+var out bytes.Buffer
+
+out.WriteString("(")
+out.WriteString({{ctx.short}}.Operator)
+out.WriteString({{ctx.short}}.Right.String())
+out.WriteString(")")
+
+return out.String()
+% @end
+
+
+
+ + + +

Output

+
+
+
+

+            type Identifier struct {
+                Token token.Token
+                Value string
+            }
+            
+            func (i *Identifier) expressionNode() {}
+            
+            func (i *Identifier) TokenLiteral() string {
+              return i.Token.Literal
+            }
+            
+            func (i *Identifier) String() string {
+              return i.Token.Literal
+            }
+            
+            type ReturnStatement struct {
+                Token token.Token
+                ReturnValue Expression
+            }
+            
+            func (rs *ReturnStatement) statementNode() {}
+            
+            func (rs *ReturnStatement) TokenLiteral() string {
+              return rs.Token.Literal
+            }
+            
+            func (rs *ReturnStatement) String() string {
+              var out bytes.Buffer
+              
+              out.WriteString(rs.Token.Literal + " ")
+              if rs.ReturnValue != nil {
+              	out.WriteString(rs.ReturnValue.String())
+              }
+              
+              out.WriteString(";")
+              
+              return out.String()
+            }
+            
+            type ExpressionStatement struct {
+                Token token.Token
+                Expression Expression
+            }
+            
+            func (es *ExpressionStatement) statementNode() {}
+            
+            func (es *ExpressionStatement) TokenLiteral() string {
+              return es.Token.Literal
+            }
+            
+            func (es *ExpressionStatement) String() string {
+              if es.Expression != nil {
+              	return es.Expression.String()
+              }
+              return ""
+            }
+            
+            type IntegerLiteral struct {
+                Token token.Token
+                Value int64
+            }
+            
+            func (il *IntegerLiteral) expressionNode() {}
+            
+            func (il *IntegerLiteral) TokenLiteral() string {
+              return il.Token.Literal
+            }
+            
+            func (il *IntegerLiteral) String() string {
+              return il.Token.Literal
+            }
+            
+            type PrefixExpression struct {
+                Token token.Token
+                Operator string
+                Right Expression
+            }
+            
+            func (pe *PrefixExpression) expressionNode() {}
+            
+            func (pe *PrefixExpression) TokenLiteral() string {
+              return pe.Token.Literal
+            }
+            
+            func (pe *PrefixExpression) String() string {
+              var out bytes.Buffer
+              
+              out.WriteString("(")
+              out.WriteString(pe.Operator)
+              out.WriteString(pe.Right.String())
+              out.WriteString(")")
+              
+              return out.String()
+            }
+
+
+
+ +

Homework: refactoring

+

+ I wanted to implement this piece-meal, with you looking on, to get a sense of the process rather than just seeing the finished thing. + As a next step, I would refactor the model out into its own file, ast_model.lua. If we did this, we would not have to pass the model (self) reference (M) from the template to the nodes() model function in ast.htt.lua.

+

+ To achieve this, we would: +

    +
  1. move the model code from ast.htt.lua to ast_model.lua
  2. +
  3. in ast.htt, import the module by adding % local model = require 'ast_model' to the top of the file
  4. +
  5. in ast.htt, change the call to nodes(M) to model.nodes()
  6. +
  7. in ast_model.lua, import the template module by adding local tpl = require '//ast.htt' to the top of the file.
  8. +
  9. in ast_model.lua, change all references to the model m.component to tpl.component.
  10. +

+
+
+
+ + + \ No newline at end of file diff --git a/htt/handbook/debug/index.html b/htt/handbook/debug/index.html new file mode 100644 index 0000000..a202d31 --- /dev/null +++ b/htt/handbook/debug/index.html @@ -0,0 +1,342 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

Debug

+

+ You will make mistakes. Here we will discuss the types of errors you will encounter and how to read the error output.

+ +

Types of errors

+

+ The types of error you will encounter are: +

    +
  1. Failure to load the initial script
  2. +
  3. Failure to compile some template file
  4. +
  5. Failure to load a lua module (whether lua code or a compiled template file)
  6. +
  7. Some error during evaluation
  8. +

+ +

How to Read Errors

+

+ We will revisit the specifics of this error later, but for now, let's cover some general information on reading the error.

+ +
+
+
+
resolved script_fpath: '/home/runner/work/htt/htt/docs/examples/debug/run-err-stx.lua'
+resolved htt_root: '/home/runner/work/htt/htt/docs/examples/debug'
+
+Error loading Lua module '//err-stx.htt':
+	err-stx.out.lua:7: 'then' expected near 'end'
+stack traceback:
+	[HTT Library]:524: in function <[HTT Library]:482>
+	[C]: in function 'require'
+	.../runner/work/htt/htt/docs/examples/debug/run-err-stx.lua:1: in main chunk
+	[C]: in function 'xpcall'
+	[HTT Library]:119: in function <[HTT Library]:101>
+
+
+
+
+ +

Resolved script fpath/htt root

+

+ script_fpath refers to the script file you used as a starting-point when calling HTT, see script file for details.

+ +

+ htt_root refers to the parent directory of the script file. This is the starting-point which is used when importing additional Lua modules or HTT template files, see HTT root for details.

+ +

References to [HTT Library]

+

+ Many lua functions in HTT are provided in addition to the Lua standard API. [HTT Library]:<number> refers to a line within the ./src/prelude.lua, which holds the definitions for this expanded standard library.

+ +

How to read the stack trace

+ The stack trace is read from bottom (outermost) to top (innermost), meaning the error originated at the line mentioned at the first like after stack traceback:. + + + +

Error Compiling HTT Template

+

+ Compile errors happen when a HTT file cannot be compiled to a Lua module because there is a syntax- or semantic error, which the compiler detected.

+ +

+ Here's a trivial example. Content (output) cannot appear outside of a component, so the first line causes a compile error:

+
+
+
+
This line causes a compile failure
+% @component one
+% local x = false
+% if x then
+% end
+% @end
+
+
+
+ +

+ These errors should be the easiest to follow. They point out the file, line- and column number. In case you want to know more, it also prints which parse state the compiler was in when encountering the error:

+ +
+
+
+
resolved script_fpath: '/home/runner/work/htt/htt/docs/examples/debug/run-err-compile.lua'
+resolved htt_root: '/home/runner/work/htt/htt/docs/examples/debug'
+
+Error compiling HTT template '//err-compile.htt'
+	File: 'err-compile.htt'
+	Line: 1
+	Column: 0
+	Type: illegal_toplevel_content
+	Reason: content must be inside a component
+stack traceback:
+	[HTT Library]:516: in function <[HTT Library]:482>
+	[C]: in function 'require'
+	...ner/work/htt/htt/docs/examples/debug/run-err-compile.lua:1: in main chunk
+	[C]: in function 'xpcall'
+	[HTT Library]:119: in function <[HTT Library]:101>
+
+
+
+
+ +

Error loading Lua module

+

+ The initial script is loaded in a special way. For all other modules, whether written in plain Lua or compiled from HTT templates, Error loading Lua module means the module contains code which is not valid Lua.

+ +

+ In this case, we wrote if x, not if x then as is required:

+
+
+
+
% @component one
+% local x = false
+% if x
+% end
+% @end
+
+
+
+ + +

+ And here is the resulting error:

+ +
+
+
+
resolved script_fpath: '/home/runner/work/htt/htt/docs/examples/debug/run-err-stx.lua'
+resolved htt_root: '/home/runner/work/htt/htt/docs/examples/debug'
+
+Error loading Lua module '//err-stx.htt':
+	err-stx.out.lua:7: 'then' expected near 'end'
+stack traceback:
+	[HTT Library]:524: in function <[HTT Library]:482>
+	[C]: in function 'require'
+	.../runner/work/htt/htt/docs/examples/debug/run-err-stx.lua:1: in main chunk
+	[C]: in function 'xpcall'
+	[HTT Library]:119: in function <[HTT Library]:101>
+
+
+
+
+ +

+ The stack trace is uninteresting in this case. We see the issue stems from a require on line 1 of the start.lua file, which is the intial script we passed to htt.

+ +

+ However, note the error, Error loading Lua module, and also note the name of the module, //err-stx.htt, which is a HTT template. For HTT templates, the module name reflects the path with // being the start of the path, relative to the HTT root directory.

+ +

+ The second thing to note is the actual error from Lua, err-stx.out.lua:7, the result of compiling err-stx.htt to Lua is saved in err-stx.out.lua, and the error in the Lua code is on line 7.

+ +

+ In this case, the actual error message from Lua, 'then' expected near 'end' is easy to follow. Other times, opening the compiled output is needed. Don't worry, the compiled output is written to be easy to follow.

+ + + +

Error during execution

+

+ Template code can cause errors. If an error happens, you can determine where by examining the stack.

+ +

+ In this example, we have two components, parent and child, both defined in //examples/debug/err-exec.htt. The parent calls child which causes a runtime error.

+ +
+
+
+
% @component parent
+% local x = false
+% if x then
+% end
+Call child {{@ child {} }}
+% @end
+
+
+
+
+
+
+
% @component child
+% error("I am raising an error")
+% @end
+
+
+
+ +

+ Note in output that we see the error itself, and before that, we can see the call-stack for components, //examples/debug/err-exec.htt.parent is the parent component, and the stack shows how it calls //examples/debug/err-exec.htt.child (child).

+ +
+
+
+
resolved script_fpath: '/home/runner/work/htt/htt/docs/examples/debug/run-err-exec.lua'
+resolved htt_root: '/home/runner/work/htt/htt/docs/examples/debug'
+
+Error during execution:
+	err-exec.out.lua:5: I am raising an error
+stack traceback:
+	err-exec.out.lua:5: in function '//err-exec.htt.child'
+	[HTT Library]:467: in field 'render'
+	err-exec.out.lua:16: in function '//err-exec.htt.parent'
+	[HTT Library]:467: in local 'render_'
+	[HTT Library]:478: in field 'render'
+	[HTT Loader]:19: in function 'render'
+	...runner/work/htt/htt/docs/examples/debug/run-err-exec.lua:2: in main chunk
+	[C]: in function 'xpcall'
+	[HTT Library]:119: in function <[HTT Library]:101>
+
+
+
+
+ +

+ +
+
+
+ + + \ No newline at end of file diff --git a/htt/handbook/modules-and-files/index.html b/htt/handbook/modules-and-files/index.html new file mode 100644 index 0000000..8226ca3 --- /dev/null +++ b/htt/handbook/modules-and-files/index.html @@ -0,0 +1,205 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

Modules and files

+

+ This covers how HTT and indeed Lua resolves and imports code and how this relates to module names. This is important both when debugging errors and laying out your projects.

+ +

Terminology

+ +

Template

+

+ A template is a <file>.htt file, written in the HTT template syntax. HTT will compile this template to a Lua module, <file>.out.lua when it is first loaded. A template may also have a companion Code file, <file>.htt.lua, whose contents is prepended to the compiled module.

+ +

Module

+

+ A module is a Lua concept and is, like a Python module, some unit of code and data. A module is typically a .lua file which would typically "export" a table of functions and data, defining the module's public API. + In the broader Lua ecosystem, modules may be implemented in C or similar by using the C API.

+ +

Component

+

+ A component is a HTT concept. It is a "mini-template", a fragment of templating which can be called and passed around as a function to be rendered inside other components. See more in the Quick Start

+ + +

Script file

+

+ By script file, we refer to the <file>.lua file which is passed to htt when running the program and which kicks off the code-generation process.

+ +

Using code from other files

+ + +

HTT root

+

+ You start the code-generation process by passing a script file to HTT, e.g. htt <file>.lua. + As the script is loaded HTT resolves the path to the script file itself, and treats the directory of that file as the HTT root. All Lua modules and HTT template files are resolved relative to the HTT root directory.

+ +

Importing Lua Modules

+

+ When importing modules, Lua uses the string in package.path to build a list of paths to try, in-order. We can see the package.path by printing it: +

+
+
+
$ cat /tmp/test.lua
+  print(package.path)
+  $ htt /tmp/test.lua
+  /tmp/?.lua;/tmp/?/init.lua;
+  
+
+
+

+ +

+ Each path in package.path is separated by ;, with ? being the placeholder which is replaced by the module name (the string argument to require).

+ +

+ If package.path is /tmp/?.lua;/tmp/?/init.lua; and we require("foo"), Lua would: +

    +
  • Try loading /tmp/foo.lua
  • +
  • (Otherwise) Try loading /tmp/foo/init.lua
  • +
  • Raise an error - could not find the module
  • +
+ +

+ If package.path is /tmp/?.lua;/tmp/?/init.lua; and we require("foo.bar"), Lua would: +

    +
  • Try loading /tmp/foo/bar.lua
  • +
  • (Otherwise) Try loading /tmp/foo/bar/init.lua
  • +
  • Raise an error - could not find the module
  • +
+ +

Importing HTT Templates

+

+ HTT extends Lua's regular require function to also work for importing HTT templates. When requiring templates, three things are different from importing regular Lua modules: + +

    +
  • We start the require string with //
  • +
  • The separator is / (instead of Lua's .), also on Windows
  • +
  • We also write the .htt file extension
  • +

+ +

+ Basically, we write what looks like a relative path, using the Unix path-separator (/) with // prefixed.

+ +
+
+
+
  -- look for "foo.htt", relative to HTT root
+  local mod = require("//foo.htt")
+  
+  -- look for "bar/foo.htt", relative to HTT root
+  local mod2 = require("//bar/foo.htt")
+  
+
+
+
+ +
+
+
+ + + \ No newline at end of file diff --git a/htt/handbook/syntax-recap/index.html b/htt/handbook/syntax-recap/index.html new file mode 100644 index 0000000..a789e1b --- /dev/null +++ b/htt/handbook/syntax-recap/index.html @@ -0,0 +1,280 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

Syntax Recap

+

+ This section is just a minimal recap of the HTT templating syntax so you have the proper vocabulary in place and can visually recognize the various parts of the syntax.

+

+ To learn how to apply this syntax and write actual templates, see the Quick Start guide.

+ +

+ Finally, you may actually want to see a condensed grammar definition. If so, click here

+ +

Literal Text

+

+ The majority of the template will probably be literal text. + Any text which is not triggering any of the syntax rules below is rendered exactly as you typed it in. This is what permits most templates to still be recognizable to people understanding the eventual output.

+ +

Lua Line

+
+
+
+
% --[[ lua code here ]]--
+
+
+
+ +

+ Lua lines begin with are lines whose first non-whitespace character is %. The remainder of the line is handled as Lua code and inserted verbatim in the compiled Template's output.

+ +

Directives

+

+ Directives are blocks with a defined start- (% @<directive type>) and end (% @end). + Directives cannot partially overlap, but code-directives can be nested inside component-directives.

+ +

Component

+
+
+
+
% @component myComponent
+... content here :)
+% @end
+
+
+
+ +

+ References to ctx within the component refers to the Lua table which holds all arguments passed to the component when called.

+ +

(Lua) Code

+

+ You can write blocks of verbatim Lua code by wrapping it in a @code directive:

+
+
+
+
% @code
+-- lua code here
+% @end
+
+
+
+ +

(Lua) Expressions

+

+ Whenever you see {{ ... }}, it is a Lua expression.

+

+ Expressions are evaluated and tostring() is called on their value and it is this value which is embedded in the output.

+ +

Component Render Call

+

+ Calls of the form {{<component> <lua-table-expr>}}.

+ +

+ For example, {{greeting {name = "John"} }} calls the component greeting with name set to John, accessible from within the component as ctx.name.

+ +

Line Continuation

+

+ Any line which starts with ~> (after optional indentation).

+ +

+ It may sound technical, but it is simple. A line continuation is just that, a continuation of the prior line.

+ + + +

Grammar

+

+ The following is a definition of the HTT grammar. +

How to read the grammar

+
    +
  • // ... this is a comment
  • +
  • <rule-name> = is the start of a rule
  • +
  • | ... - is the start of one (of possibly several) definitions of the rule
  • +
  • . ... - means the definition spans multiple lines and this is the next one
  • +
  • <rule>* means 0 or more repetitions of rule
  • +
  • <rule>+ means 1 or more repetitions of rule
  • +
  • <rule>? means zero or 1 repetitions of rule
  • +
  • '...' (anything in single-quotes) is a literal value
  • +

+

Grammar

+
+
+
+
top =
+    | lua-line
+    | component
+    | code
+
+  lua-line =
+    | ws* '%' lua nl
+
+  code =
+    | code-open nl
+    . lua nl
+    . code-close nl
+
+  code-open =
+    | ws* '%' ws+ '@code'
+
+  code-close = 
+    | ws* '%' ws+ '@end'
+
+  component =
+    | component-open nl
+    . component-line
+    . component-close nl
+
+  component-open =
+    | ws* '%' ws+ '@component' component_name
+
+  component_name =
+    | [a-zA-Z_][a-zA-Z0-9_]*
+
+  component-line =
+    | lua-line
+    | text-line
+    | code
+
+  component-close =
+    | ws* '%' ws+ '@end'
+
+  text-line =
+    | ws* line-continuation? element* nl
+
+  line-continuation =
+    | '~>'
+
+  element =
+    | text
+    | expression
+    | component-call
+
+  // here, 'lua' must be a single-line expression.
+  expression =
+    | '{{' lua '}}'
+
+  // here 'lua-table-expr' is some single-line lua expression
+  // which evaluates to a Lua table.
+  component-call =
+    | '{{@' ws* component_name ws+ lua-table-expr '}}'
+
+  ws =
+    | \r
+    | \t
+    | ' '
+
+  nl =
+    | \r? \n
+
+
+
+
+
+
+
+ + + \ No newline at end of file diff --git a/htt/index.html b/htt/index.html new file mode 100644 index 0000000..66dc7ca --- /dev/null +++ b/htt/index.html @@ -0,0 +1,132 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

What is HTT?

+

+ HTT is a code-generator. It is built to help you generate code or configuration files from some high-level description which you either write yourself or get from elsewhere.

+ +

What can I do with code generation?

+

+ You could generate repetitive code, such as a slew of AST types in Go, dataclass-like classes in Python, a series of functions for manipulating bits without using bitfields in C and so on. The possibilities are everywhere.

+ +

+ But it is not limited to code. This site is generated using HTT. It embeds code examples from actual files, and it runs them, captures their output and embeds that also. It also generates the menu-structure and places output in a folder structure which enables user-friendly URLs.

+ +

+ Code generation is also great whenever crossing language- or service/application boundaries. + Think of programs written in different languages exchanging messages with code generated from a Protobuf description. Or of clients, whose code for interacting with a web-service is generated from its Swagger API description.

+ +

+ Finally, code generation is also an excellent way to address shortcomings of configuration languages like Ansible, terraform, Nix and so on. + If your DSL files become very long, verbose or difficult to read, perhaps it is time to do some pre-processing in a more capable language, like Lua, and generate simpler DSL files instead.

+ +
+
+
+ + + \ No newline at end of file diff --git a/htt/quick-start/index.html b/htt/quick-start/index.html new file mode 100644 index 0000000..0daa658 --- /dev/null +++ b/htt/quick-start/index.html @@ -0,0 +1,482 @@ + + + + + + HTT Documentation + + + + + + +
+
+ +

HTT Documentation

+
+
+ + +
+ +
+ +
+
+ +
+
+ + +
+
+

Quick Start

+ +

Hello World

+

HTT takes a single Lua script as its argument, from here you can generate as many files as you want by calling the render function.

+

Let's create a script, test-htt.lua:

+ +
+
+
+
local tpl = require("//helloworld.htt")
+
+render(tpl.helloWorld, "result.txt")
+
+
+
+ + + + + + Let's then fill out the template file, helloworld.htt: + +
+
+
+
% @component helloWorld
+Hello, John!
+% @end
+
+
+
+ + If you now run the command htt test-htt.lua, then the result.txt file will look like this: + +
+
+
+
Hello, John!
+
+
+
+ +

To Summarize

+
    +
  1. We wrote a template file, helloworld.htt, and defined a component, helloWorld, inside it
  2. +
  3. We wrote a Lua script, test-htt.lua, which uses the render call to render a component, helloWorld, to a file: result.txt.
  4. +
+ +

Using Lua inside templates

+

HTT uses plain Lua for all logic inside of templates. Any line which starts with % is taken to be a line of Lua code injected verbatim into the compiled template.

+ +

+ +

Looping with Lua

+

We can use Lua for loops to repeat a block of output like so:

+ +
+
+
+
% for i = 1, 3 do
+hello!
+% end
+
+
+
+
+
+
+
hello!
+            hello!
+            hello!
+
+
+
+ +

Conditional rendering with Lua

+

We can use Lua if-statements to decide whether to render a block of output or not:

+ +
+
+
+
% if 1 == 2 then
+Math is broken!
+% else
+Phew! Math works
+% end
+
+
+
+ + If we render this component, we get: +
+
+
+
Phew! Math works
+
+
+
+ +

Using variables in output

+

You can use {{ ... }} to embed the values of variables or other expressions in the output:

+
+
+
+
% local name = "Peter"
+Hello, {{name}}!
+2 + 2 is {{2 + 2}}
+
+
+
+
+
+
+
Hello, Peter!
+            2 + 2 is 4
+
+
+
+ + + + +

Components

+

Components are the unit of abstraction. Complex outputs should be built by composing smaller components into larger ones.

+ +

Calling components from within components

+

To call a component, we use this syntax: {{@ component ctx }}. component is any component and ctx is the context we want to pass on to the component. The context should be a Lua table and it is the way to pass arguments to the component (including other components, if you want!).

+ +
+

Given this component:

+
+
+
+
% @component child
+Sincerely hello
+  - Child
+% @end
+
+
+
+ +

Let's call it from another component.

+ +
+
+
+
% @component parent
+let's call the child:
+{{@ child {} }}
+% @end
+
+
+
+ +

The output becomes:

+
+
+
+
let's call the child:
+            Sincerely hello
+              - Child
+
+
+
+ +

Providing arguments to Components

+

When introducing the syntax to call a component, {{@ component ctx }}, we touched on the ctx being a Lua table.

+ +

We first define a component which uses the argument name, if provided. Note that all arguments passed to a component are exposed via the ctx variable. ctx.name is simply the Lua way of accessing the attribute name on the ctx table.

+
+
+
+
% @component helloName
+Hello, {{ctx.name or "John Doe"}}!
+% @end
+
+
+
+ +

Calling component without name argument

+

Calling the component without no name:

+
+
+
+
{{@ helloName {} }}
+
+
+
+ +

Produces the following:

+
+
+
+
Hello, John Doe!
+
+
+
+ +

Calling component with name argument

+

Calling the component without no name:

+
+
+
+
{{@ helloName {name = "Peter"} }}
+
+
+
+ +

Produces the following:

+
+
+
+
Hello, Peter!
+
+
+
+ +

Indentation and Components

+ A focus of HTT has been to get indentation "right". Before we summarize how it works, here's an expanded example: + +
+
+
+
% @component child
+Sincerely hello
+  - Child
+% @end
+
+
+
+ +
+
+
+
% @component middleChild
+--middle child
+  {{@ child {} }}
+% @end
+
+
+
+ +
+
+
+
% @component callWithIndentation
+{{@ child {} }}
+    {{@ middleChild {} }}
+    {{@ child {} }}
+% @end
+
+
+
+ +

Rendering this out gives this output:

+ +
+
+
+
Sincerely hello
+              - Child
+                --middle child
+                  Sincerely hello
+                    - Child
+                Sincerely hello
+                  - Child
+
+
+
+ + + + + +

Line Continuations

+

There will be times, usually when mixing output and Lua lines, where you want to build up a single line of output across multiple lines in the template.

+ +

If a line starts with ~> (after whitespace), then the line is taken to be a continuation of the preceding line of output.

+ +

Simple Example

+
+
+
+
hello
+~>world!
+hello
+~> world...
+
+
+
+

This renders out to:

+
+
+
+
helloworld!
+            hello world...
+
+
+
+ + +

Realistic Example

+

The following example builds up an array where elements are separated by , . Another example would be rendering function arguments or parameters.

+
+
+
+
% local sep = ctx.separator or ","
+[
+% for i, v in ipairs(ctx.lst) do
+% local is_last = (i == #ctx.lst)
+~> "{{v}}"{{not is_last and sep or ""}}
+% end
+~> ]
+
+
+
+

This renders out to:

+
+
+
+
[ "one", "two", "three" ]
+
+
+
+ +

The line printing the separator uses a trick in Lua to simulate a ternary-if:

$test and $if-truthy or $otherwise.
+ Alternatively, we could have used an if-block instead.

+
+
+
+ + + \ No newline at end of file