Add off-chain-data go client application

[twoGiants]

Off Data Go Application

This PR contains an implementation of the off-chain data store sample in go using the block event listening capability of the Fabric Gateway client API. The issue reference is here in the Fabric Gateway SDK repository.

It follows the same procedure and uses mostly the same naming as the off-chain data typescript sample here. There are some differences in the go package structure compared to the typescript modules and I added a couple of tests to the parser which helped me better understand the unpacking of the block data structure. The Go implementation is more similar to the Java version. More about that below in the Differences section.

@bestbeforetoday gave me this useful link where I found the iterator_test.go to be helpful and this link where the "Endorser Transaction" comments were helpful.

Summary

• created project directory, app.go, getAllAssets.go, transact.go and listen.go files
• reused and changed the logic for connect.go from the private data application
• implemented the asset transfer basic contract in the contract package
• implemented getAllAssets.go and transact.go
• implemented the block parser and all the supporting data structures
• implemented the block processor and all the supporting data structures
• implemented listen.go
• updated README.md with the command to run the go application
• updated script which runs the application in the Github Actions workflow
• manual tests

Differences

• I didn't use any interfaces. I removed them after the suggestion of @bestbeforetoday.
• I did not implement any methods which are not used anywhere, like toProto or getSignatureHeader.
• The block parser with all it's underlying structs (which are mostly private) is in the parser package.
• There are some simple tests for the cache utility and the block unwrapping.
• The block processor with it's underlying struct transaction (which is private) is in the processor package.
• The store is in the store package and because it resembles a bit a flat file db I called it flatFile.go
• The utilities are in the utils package.
• In all the additional packages I followed the encapsulation rule: if it's not referenced outside the package it's private.

Additional Information

I process the block events in a go routine and use context to stop the processing. Instead of panicking everywhere down the line where an error can occur I handle the errors by wrapping them, returning them, log them in the go routine, then return and shutdown gracefully. Panicking results in the context, gateway and checkpointer not getting closed.

Questions

1. Here is a TODO proposing to simplify the processor.transaction data structure.
2. And here is a TODO proposing to encapsulate the final unwrapping of the kvWrite in the parser instead of having it in transaction.writes.
3. I assume I should squash everything to one commit? It might be a big commit message.

[bestbeforetoday]

First thing, it runs great for me so this looks nicely done. Thank you! Sorry for large number of comments. This is just polishing the material to make it a really clean sample.

I would be careful to try and keep all the code that listens for blocks, navigates to their transactions and passes the read/write sets to some kind of store close together. Perhaps all in listen.go. This is the key learning point of the sample so it needs to be easy to access and navigate. The block parser should be separate (as you've done) so it is a reusable package on its own. The store implementation can also be a little saperate to avoid cluttering the key listener code. I might put it in a different file within the same package though, just as private (lowercase) content.

The Application section of off_chain_data/README.md should be updated to include references to key parts of the Go implementation, similar to the existing TypeScript and Java entries.

To catch some issues, it might be worth running this command in the application-go directory:

go run honnef.co/go/tools/cmd/staticcheck@latest -f stylish -checks all ./...

To avoid specific checks (like having package-level godoc), you can exclude specific rules. For example:

go run honnef.co/go/tools/cmd/staticcheck@latest -f stylish -checks 'all,-ST1000' ./...

[bestbeforetoday]

The clientConnection parameter name can be omitted here. Your choice which you find clearer though.

Suggested change

	var allCommands = map[string]func(clientConnection *grpc.ClientConn){
	var allCommands = map[string]func(*grpc.ClientConn){

[twoGiants]

Omitted.

[bestbeforetoday]

The returned error value should be checked to ensure the write was successful.

Suggested change

	b.checkpointer.CheckpointTransaction(blockNumber, transactionId)
	if err := b.checkpointer.CheckpointTransaction(blockNumber, transactionId); err != nil {
	return err
	}

[twoGiants]

Thx. Done.

[bestbeforetoday]

The returned error value should be checked to ensure the write was successful.

Suggested change

	b.checkpointer.CheckpointBlock(blockNumber)
	return b.checkpointer.CheckpointBlock(blockNumber)

[twoGiants]

Thx. Done.

[bestbeforetoday]

This is only used by the transact command logic. It is not really relevant to the asset type definition. I would move it to the transact command implementation.

[bestbeforetoday]

This is only used by the transact command logic. It is not really relevant to the asset type definition. I would move it to the transact command implementation.

[twoGiants]

I am also using it in the parser/block_test.go. I think keeping the Asset construction next to the asset is cohesive or wdys? But being that it's only used in transact I can see your point also.

I will move it to transact, it's no problem and use a simpler construction in the test.

[bestbeforetoday]

The standard library has a function to join an array of strings. Using this simplifies this type enough that I would consider removing it entirely and just building a generic error using fmt.Errorf at the point of use.

Suggested change

	return fmt.Sprintf(format, t.txId, t.blockNumber, t.blockTxIdsJoinedByComma())
	}
	func (t *txIdNotFoundError) blockTxIdsJoinedByComma() string {
	result := ""
	for index, item := range t.blockTxIds {
	if len(t.blockTxIds)-1 == index {
	result += item
	} else {
	result += item + ", "
	}
	}
	return result
	}
	return fmt.Sprintf(format, t.txId, t.blockNumber, strings.Join(t.blockTxIds, ", "))
	}

[bestbeforetoday]

See https://go.dev/wiki/CodeReviewComments#declaring-empty-slices

Suggested change

	return []byte{}, fmt.Errorf("in GetAllAssets: %w", err)
	return nil, fmt.Errorf("in GetAllAssets: %w", err)

[bestbeforetoday]

transactions is never used

[bestbeforetoday]

I'm not sure of the reason for using a wait group for a single goroutine that you then wait on before continuing. Maybe this can just be inline code?

It should not be necessary to check the context while reading. If the context used for the BlockEvents call is closed, the returned channel should also get closed, which would mean the range loop will exit.

Suggested change

	var wg sync.WaitGroup
	wg.Add(1)
	go func() {
	defer wg.Done()
	for blockProto := range blocks {
	select {
	case <-ctx.Done():
	return
	default:
	blockProcessor := processor.NewBlock(
	parser.ParseBlock(blockProto),
	checkpointer,
	store.ApplyWritesToOffChainStore,
	channelName,
	)
	if err := blockProcessor.Process(); err != nil {
	fmt.Println("\033[31m[ERROR]\033[0m", err)
	return
	}
	}
	}
	}()
	wg.Wait()
	for blockProto := range blocks {
	blockProcessor := processor.NewBlock(
	parser.ParseBlock(blockProto),
	checkpointer,
	store.ApplyWritesToOffChainStore,
	channelName,
	)
	if err := blockProcessor.Process(); err != nil {
	fmt.Println("\033[31m[ERROR]\033[0m", err)
	return
	}
	}

[bestbeforetoday]

By convention Go package names are all lowercase. See https://go.dev/blog/package-names