Initial import

LoxiGen is the work of several developers, not just myself.

.gitignore

@@ -0,0 +1,6 @@
+*.pyc
+*~
+target_code
+local
+.loxi_ts.*
+.loxi_gen_files

INTERNALS

@@ -0,0 +1,76 @@
+
+Here are a few notes about the LOXI processing flow.
+
+Currently there are two pieces of input for each version to be supported.
+
+(1) The original openflow.h header file.  This is parsed to extract
+identifiers such as #defines and enum definitions.  These are in the
+'canonical' directory.
+
+(2) A specially processed list of structs derived from the original
+openflow.h header file.  These are the structs that represent the
+protocol on the wire with the following minor modifications:
+** ofp_header structures instances are replaced by their contents
+** Arrays are replaced with the syntax 'data_type[length] idenitifier'.
+** Lists of objects are called out explicitly as 'list(data_type) identifier'
+** Match structures are renamed to be version specific
+** Each flavors of a flow modify (add, modify, modify strict, delete
+and delete strict) are called out as different objects
+** Each action type (for instance) is called out as its own type.
+
+Copyright 2012, Big Switch Networks, Inc.
+
+Enumerations/defines give semantic values for two contexts:
+
+* Internal management of objects, for example, the particular values that
+indicate a message is an Echo message or an action is an output action.
+These values, like the wire format, are generally not of interest to 
+the users of LOXI.
+
+* External representation of information. These are values which users of
+LOXI need to know about, at least through an identifier.  Examples include
+OFP_TCP_PORT, OFP_MAX_TABLE_NAME_LEN or OFPP_MAX.  
+
+In general, processing proceeds by:
+
+(1) Extracting information from each version's input files.
+
+(2) Unifying the information across all versions, allowing the
+identification of commonalities and differnces.
+
+(3) Calling the language specific generation routines for each
+target file.  The list of files to generate and the map from 
+file to generating function is given in the language specific
+Python file such as lang_c.py at the top level.
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The code layout is as follows (explanations below):
+
+BigCode/Modules/
+    LoxiGen/
+        Makefile
+        loxigen.py      Entry point executable
+        of_g.py         Global variables   
+        lang_c.py, ...  Language specific
+        loxi_front_end/ Python functions for processing input
+        loxi_utils/     General utility functions
+        canonical/      openflow.h header files
+            openflow.h-<of-version>
+        openflow_input/ pre-processed openflow.h input
+            structs-<of-version>
+        c_gen/          Python functions for C code generation
+        c_template/     Template including non-autogen files
+        utest/          Simple Python scripts to test functions
+
+For C code generation, the output is placed in the BigCode module format.
+First, the C template directory is copied over to a target directory.
+Then the automatically generated files are created and placed in the
+proper locations in the target directory.  Then the result is tarred
+up for overlay onto another location.
+
+To test the code locally, the target file is untarred into a local
+directory and a special make file (in c_gen/Makefile.local) is copied
+into the local directory.
+
+

LoxiGen.mk

@@ -0,0 +1,39 @@
+# Copyright 2013, Big Switch Networks, Inc.
+#
+# LoxiGen is licensed under the Eclipse Public License, version 1.0 (EPL), with
+# the following special exception:
+#
+# LOXI Exception
+#
+# As a special exception to the terms of the EPL, you may distribute libraries
+# generated by LoxiGen (LoxiGen Libraries) under the terms of your choice, provided
+# that copyright and licensing notices generated by LoxiGen are not altered or removed
+# from the LoxiGen Libraries and the notice provided below is (i) included in
+# the LoxiGen Libraries, if distributed in source code form and (ii) included in any
+# documentation for the LoxiGen Libraries, if distributed in binary form.
+#
+# Notice: "Copyright 2013, Big Switch Networks, Inc. This library was generated by the LoxiGen Compiler."
+#
+# You may not use this file except in compliance with the EPL or LOXI Exception. You may obtain
+# a copy of the EPL at:
+#
+# http://www.eclipse.org/legal/epl-v10.html
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# EPL for the specific language governing permissions and limitations
+# under the EPL.
+
+#
+# Static for LoxiGen
+#
+LOXIGEN_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
+
+LoxiGen:
+	$(MAKE) -C $(LOXIGEN_DIR) all
+
+ALL_TARGETS += LoxiGen
+
+DEPENDMODULES_XHEADER_EXCLUDES += LoxiGen
+

Makefile

@@ -0,0 +1,91 @@
+# Copyright 2013, Big Switch Networks, Inc.
+#
+# LoxiGen is licensed under the Eclipse Public License, version 1.0 (EPL), with
+# the following special exception:
+#
+# LOXI Exception
+#
+# As a special exception to the terms of the EPL, you may distribute libraries
+# generated by LoxiGen (LoxiGen Libraries) under the terms of your choice, provided
+# that copyright and licensing notices generated by LoxiGen are not altered or removed
+# from the LoxiGen Libraries and the notice provided below is (i) included in
+# the LoxiGen Libraries, if distributed in source code form and (ii) included in any
+# documentation for the LoxiGen Libraries, if distributed in binary form.
+#
+# Notice: "Copyright 2013, Big Switch Networks, Inc. This library was generated by the LoxiGen Compiler."
+#
+# You may not use this file except in compliance with the EPL or LOXI Exception. You may obtain
+# a copy of the EPL at:
+#
+# http://www.eclipse.org/legal/epl-v10.html
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# EPL for the specific language governing permissions and limitations
+# under the EPL.
+
+# Available targets: all, c, python, clean
+
+# This Makefile is just for convenience. Users that need to pass additional
+# options to loxigen.py are encouraged to run it directly.
+
+# Where to put the generated code.
+LOXI_OUTPUT_DIR = loxi_output
+
+# Generated files depend on all Loxi code and input files
+LOXI_PY_FILES=$(shell find \( -name loxi_output -prune \
+                             -o -name templates -prune \
+                             -o -true \
+                           \) -a -name '*.py')
+LOXI_TEMPLATE_FILES=$(shell find */templates -type f -a \
+                                 \! \( -name '*.cache' -o -name '.*' \))
+INPUT_FILES = $(wildcard openflow_input/*) $(wildcard canonical/*)
+
+all: c python
+
+c: .loxi_ts.c
+
+.loxi_ts.c: ${LOXI_PY_FILES} ${LOXI_TEMPLATE_FILES} ${INPUT_FILES}
+	./loxigen.py --install-dir=${LOXI_OUTPUT_DIR} --lang=c
+	touch $@
+
+python: .loxi_ts.python
+
+.loxi_ts.python: ${LOXI_PY_FILES} ${LOXI_TEMPLATE_FILES} ${INPUT_FILES}
+	./loxigen.py --install-dir=${LOXI_OUTPUT_DIR} --lang=python
+	touch $@
+
+clean:
+	rm -rf loxi_output # only delete generated files in the default directory
+	rm -f loxigen.log loxigen-test.log .loxi_ts.c .loxi_ts.python
+
+debug:
+	@echo "LOXI_OUTPUT_DIR=\"${LOXI_OUTPUT_DIR}\""
+	@echo
+	@echo "LOXI_PY_FILES=\"${LOXI_PY_FILES}\""
+	@echo
+	@echo "LOXI_TEMPLATE_FILES=\"${LOXI_TEMPLATE_FILES}\""
+	@echo
+	@echo "INPUT_FILES=\"${INPUT_FILES}\""
+
+check:
+	@echo Sending output to loxigen-test.log
+	cd utest && \
+		./identifiers_test.py > ../loxigen-test.log && \
+		./c_utils_test.py >> ../loxigen-test.log && \
+		./of_h_utils_test.py >> ../loxigen-test.log
+	PYTHONPATH=. ./utest/test_parser.py
+
+pylint:
+	pylint -E ${LOXI_PY_FILES}
+
+.PHONY: all clean debug check pylint c python
+
+ifdef BIGCODE
+# Internal build system compatibility
+MODULE := LoxiGen
+LOXI_OUTPUT_DIR = ${BIGCODE}/Modules
+modulemake:
+.PHONY: modulemake
+endif

README.md

@@ -0,0 +1,41 @@
+Introduction
+============
+
+LoxiGen is a tool that generates OpenFlow protocol libraries for a number of
+languages. It is composed of a frontend that parses wire protocol descriptions
+and a backend for each supported language (currently C and Python, with Java on
+the way).
+
+
+Usage
+=====
+
+You can run LoxiGen directly from the repository. There's no need to install it,
+and it has no dependencies beyond Python 2.6+.
+
+To generate the libraries for all languages:
+
+```
+make
+```
+
+To generate the library for a single language:
+
+```
+make c
+```
+
+The currently supported languages are `c` and `python`.
+
+The generated libraries will be under the `loxi_output` directory. This can be
+changed with the `LOXI_OUTPUT_DIR` environment variable when using the Makefile.
+
+Each generated library comes with its own set of documentation in the standard
+format for that language. Please see that documentation for more details on
+using the generated libraries.
+
+Contributing
+============
+
+Please fork the repository on GitHub and send us a pull request. You might also
+be interested in the INTERNALS file which has notes about how LoxiGen works.

TODO

@@ -0,0 +1,17 @@
+
+
+For lists of uint32 and uint8, the easy approach was taken which was to
+add OF types (wrappers) for instances of these types.  This makes some
+things much easier, but makes it awkward, for example, to simply append
+a 32-bit value to the end of a list.  Some helper functions should be
+added to make this easier.  (Maybe?  Seems to be working so far.)
+
+With the added support for auto declaring var len arrays indexed by
+version, lots of trailing commas have been added.
+
+Add type and length support for action_id classes.
+
+Table feature prop uses 0xfffe instead of 0xffff as the experimenter value.
+
+Support TABLE_FEATURE_PROP_EXPERIMENTER/_MISS
+

c_gen/Makefile.local

@@ -0,0 +1,91 @@
+# Copyright 2013, Big Switch Networks, Inc.
+#
+# LoxiGen is licensed under the Eclipse Public License, version 1.0 (EPL), with
+# the following special exception:
+#
+# LOXI Exception
+#
+# As a special exception to the terms of the EPL, you may distribute libraries
+# generated by LoxiGen (LoxiGen Libraries) under the terms of your choice, provided
+# that copyright and licensing notices generated by LoxiGen are not altered or removed
+# from the LoxiGen Libraries and the notice provided below is (i) included in
+# the LoxiGen Libraries, if distributed in source code form and (ii) included in any
+# documentation for the LoxiGen Libraries, if distributed in binary form.
+#
+# Notice: "Copyright 2013, Big Switch Networks, Inc. This library was generated by the LoxiGen Compiler."
+#
+# You may not use this file except in compliance with the EPL or LOXI Exception. You may obtain
+# a copy of the EPL at:
+#
+# http://www.eclipse.org/legal/epl-v10.html
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# EPL for the specific language governing permissions and limitations
+# under the EPL.
+
+# Makefile to test LOCI generated files locally
+# Assumes code is in BigCode layout
+# Normally the loci and locitest code is put into a directory 'local' with
+# this make file copied to its top level.
+
+LOCI_SOURCES:=$(wildcard Modules/loci/module/src/*.c)
+LOCI_OBJECTS:=$(LOCI_SOURCES:.c=.o)
+LOCI_HEADERS:=$(wildcard Modules/loci/module/inc/loci/*.h)
+LOCI_HEADERS+=$(wildcard Modules/loci/module/src/*.h)
+
+LOCITEST_SOURCES:=$(wildcard Modules/locitest/module/src/*.c)
+LOCITEST_OBJECTS:=$(LOCITEST_SOURCES:.c=.o)
+LOCITEST_HEADERS:=$(wildcard Modules/locitest/module/inc/locitest/*.h)
+LOCITEST_HEADERS+=$(wildcard Modules/locitests/module/src/*.h)
+
+LOCITEST_MAIN:=Modules/locitest/utest/main.c
+
+ALL_SOURCES:=${LOCITEST_SOURCES} ${LOCI_SOURCES}
+ALL_OBJECTS:=$(ALL_SOURCES:.c=.o)
+ALL_HEADERS:=${LOCITEST_HEADERS} ${LOCI_HEADERS}
+
+CFLAGS:=-Wall -g -I Modules/loci/module/inc -I Modules/locitest/module/inc -O0
+
+all: test
+
+test: locitest
+	./locitest
+
+%.o: %.c ${ALL_HEADERS}
+	gcc -c ${CFLAGS} $< -o $@
+
+%.E: %.c ${ALL_HEADERS}
+	gcc -E ${CFLAGS} $< -o $@
+
+libloci.a: ${LOCI_OBJECTS}
+	ar rcu libloci.a ${LOCI_OBJECTS}
+	ranlib libloci.a
+
+liblocitest.a: ${LOCITEST_OBJECTS}
+	ar rcu liblocitest.a ${LOCITEST_OBJECTS}
+	ranlib liblocitest.a
+
+# Test executable
+locitest: ${LOCITEST_MAIN} libloci.a liblocitest.a
+	gcc $< ${CFLAGS} -l locitest -l loci -L . -o $@
+
+show:
+	@echo ALL_SOURCES ${ALL_SOURCES}
+	@echo ALL_OBJECTS ${ALL_OBJECTS}
+	@echo ALL_HEADERS ${ALL_HEADERS}
+
+help:
+	@echo "Run loci unit tests locally"
+
+clean:
+	find . -name '*.o' | xargs rm -f
+	find . -name '*.E' | xargs rm -f
+	rm -f libloci.a liblocitest.a locitest
+
+# TBD
+doc: ${ALL_HEADERS}
+	doxygen Doxyfile
+
+.PHONY: all test lib clean show doc

c_gen/__init__.py

@@ -0,0 +1,26 @@
+# Copyright 2013, Big Switch Networks, Inc.
+#
+# LoxiGen is licensed under the Eclipse Public License, version 1.0 (EPL), with
+# the following special exception:
+#
+# LOXI Exception
+#
+# As a special exception to the terms of the EPL, you may distribute libraries
+# generated by LoxiGen (LoxiGen Libraries) under the terms of your choice, provided
+# that copyright and licensing notices generated by LoxiGen are not altered or removed
+# from the LoxiGen Libraries and the notice provided below is (i) included in
+# the LoxiGen Libraries, if distributed in source code form and (ii) included in any
+# documentation for the LoxiGen Libraries, if distributed in binary form.
+#
+# Notice: "Copyright 2013, Big Switch Networks, Inc. This library was generated by the LoxiGen Compiler."
+#
+# You may not use this file except in compliance with the EPL or LOXI Exception. You may obtain
+# a copy of the EPL at:
+#
+# http://www.eclipse.org/legal/epl-v10.html
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# EPL for the specific language governing permissions and limitations
+# under the EPL.