Addition of new API functions, new encryption modes (AES-CTR, GCM), unit tests and CI tests with refactoring of code.

[Jakio815]

Please merge with iotauth/iotauth#38.

0. Introducing new APIs and examples.

This PR adds new API functions and an example of encryption and decryption of files in block chunks. Also adds unit tests, CI tests. Finally, major refactoring done on the entire code base.

Here is a summary of this PR.

• New APIs
  • New API functions to encrypt/decrypt buffers with/without malloc().
  • New API functions for saving and loading session keys.
  • Enable different encryption modes, and hmac_mode.
• New file_block_encrypt example
• New Unit tests
• New CI tests
• Refactoring
  • Move all examples under examples/
  • Refactoring on code base.

1. New APIs

1. Initializes empty session_key_list.

// Mallocs session_key_list_t and the session_key_t as much as the MAX_SESSION_KEY.
session_key_list_t *init_empty_session_key_list(void);

2. Connect to entity_server using the session key. This function can be called after the connect() function, and uses the user's socket.

SST_session_ctx_t *secure_connect_to_server_with_socket(session_key_t *s_key, int sock);

3. Checks the message header if it is SECURE_COMM_MSG, and fills the buffer with the received message. Input is the connected socket, pointer of the buffer, and the given buffer's length (not the received message length.)

int read_secure_message(int socket, unsigned char *buf, unsigned int buf_length)

The four functions below are for encrypting and decrypting buffers with the session key.
4, 5. These functions encrypt/decrypt the given plaintext/ciphertext with the given session key. It mallocs a buffer for the encrypted/decrypted result, and returns the double pointer of the encrypted/decrypted buffer.

int encrypt_buf_with_session_key(session_key_t *s_key, unsigned char *plaintext,
                                 unsigned int plaintext_length,
                                 unsigned char **encrypted,
                                 unsigned int *encrypted_length);
int decrypt_buf_with_session_key(session_key_t *s_key, unsigned char *encrypted,
                                 unsigned int encrypted_length,
                                 unsigned char **decrypted,
                                 unsigned int *decrypted_length);

6, 7. Unlike the function above, they do not allocate memory, the user should provide the buffer with enough length.

int encrypt_buf_with_session_key_without_malloc(
    session_key_t *s_key, unsigned char *plaintext,
    unsigned int plaintext_length, unsigned char *encrypted,
    unsigned int *encrypted_length);
int decrypt_buf_with_session_key_without_malloc(
    session_key_t *s_key, unsigned char *encrypted,
    unsigned int encrypted_length, unsigned char *decrypted,
    unsigned int *decrypted_length);

8, 9. Save/Load the session_key_list to the file_path.

int save_session_key_list(session_key_list_t *session_key_list, const char *file_path);
int load_session_key_list(session_key_list_t *session_key_list, const char *file_path);

10,11. Save/Load the session_key_list with a salted password.

int save_session_key_list_with_password(session_key_list_t *session_key_list,
                                        const char *file_path,
                                        const char *password,
                                        unsigned int password_len,
                                        const char *salt,
                                        unsigned int salt_len);
int load_session_key_list_with_password(session_key_list_t *session_key_list,
                                        const char *file_path,
                                        const char *password,
                                        unsigned int password_len,
                                        const char *salt,
                                        unsigned int salt_len);

The two below are some useful helper functions.
12. Reads the session key id buffer and saves it in an unsigned integer. This is needed when the user wants to save the session_key id as an int value.

unsigned int convert_skid_buf_to_int(unsigned char *buf, int byte_length);

13. Generates a random nonce. This is used for the user not to directly #include OpenSSL libraries.

void generate_random_nonce(int length, unsigned char *buf);

2. New example of file block encryption.

This example tests encrypting multiple buffers using different session keys, and test getting the session key by the session key ID, and decrypting them.

Encrypting part. block_writer.c

1. Create random key_values, with a size between 56~144 bytes.
2. The key_values are appended until the total size is 32 kbytes.
3. When the next key_value does not fit to the maximum block size(32kybtes), the leftover size is filled with zero-paddings. The leftover buffer will be used in the next block.
4. The block is now 32kbytes, and it is encrypted with a session key.
5. 1~5 is repeated, appending 10 blocks, making a single file.
6. 5 is repeated, making three separate files, encrypted'i'.txt. Each file uses different session keys.
7. To test if the encryption decryption worked prorperly, we save plaintext'i'.txt, which is the blocks not encrypted.
8. The metadata is saved inside encrypted_file_metadata.dat. Same with plaintext_file_metadata.dat. It saves the used session key id for the file.

Decrypt and Comparing part. block_reader.c

7. Loads the metadata saved.
8. Requests the session key corresponding to the session key id.
9. Read the encrypted'i'.txt and decrypt it with the requested session key.
10. Compare it with the read plaintext'i'.txt, and check if it's decrypted properly.

For compile details, please check the README.md

3. Refactoring encryption and decryption functions.

Change return values to enable error checking for functions.

The original code just returns the pointer of the encrypted or decrypted buffers. However, it does not do any error checking.
Now, the return values are the success and failure of the encryption and decryption.
0 stands for success, 1 for failure.

Removing memcpy() inside symmetric_encrypt_authenticate() and symmetric_decrypt_authenticate()

The original code does many unnecessary memcpy(). All of them are removed.

Add functions to encrypt/decrypt buffers with/without malloc()

We now provide API functions that encrypt/decrypt buffers. Also, The users can select to use functions with or without malloc().
These are examples using the two functions.

unsigned int encrypted_length;
unsigned char *encrypted;
encrypt_buf_with_session_key(s_key, plaintext, plaintext_length, encrypted, &encrypted_length);

unsigned int estimate_encrypted_length = get_expected_encrypted_total_length();
unsigned char encrypted[estimate_encrypted_length];
unsigned int encrypted_length;
encrypt_buf_with_session_key_without_malloc(s_key, plaintext, plaintext_length, encrypted, &encrypted_length);

4. Detailed changes on each file.

c_common.c

• Add read_from_socket() write_from_socket() with better error handlings.
• Add check_SECURE_COMM_MSG_type() to not #define the SECURE_COMM_MSG in c_api.c.
• Fix connect_as_client() to retry connect().
• parse_received_message() : Add AUTH_ALERT handling.

c_common.h

• Add descriptions for the new functions read_from_socket()``write_from_socket() check_SECURE_COMM_MSG_type()

load_config.c

• Add ENCRYPTION_MODE and HMAC_MODE
• load_config():
  • Better file pointer handling
  • Better purpose_count handling. This is for multiple purposes in the config.
  • Add Encryption mode and HMAC mode handling.
• free_config_t: Add free() for the config itself.

load_config.h

• Change the #define values to an enum config_type_t.

c_crypto.c

• Better free() and error handlings.
• digest_message_SHA_256(): Does not malloc memory anymore. Buffer should be assigned before called.
• Change AES_CBC_128_encrypt() to encrypt_AES(), and AES_CBC_128_decrypt() to decrypt_AES()
  • Make other encryption/decryption modes available (CTR, GCM).
• Newly added function:
  • get_EVP_CIPHER(): Set the EVP_CIPHER based on the encryption mode.
  • get_expected_encrypted_total_length(), get_expected_decrypted_maximum_length():
    • Returns the estimated length of the encrypted/decrypted message. This is used to not malloc() memory.
  • create_salted_password_to_32bytes(): Create a 32 byte digested password using the salt.
• Refactored functions:
  • symmetric_encrypt_authenticate() / symmetric_decrypt_authenticate()
    • Encrypt buffer, and also add hmac, depending on encryption mode and hmac_mode.
    • Internally uses get_symmetric_encrypt_authenticate_buffer() and get_symmetric_decrypt_authenticate_buffer()
  • symmetric_encrypt_authenticate_without_malloc() / symmetric_decrypt_authenticate_without_malloc()
    • Same with symmetric_encrypt_authenticate() but does not malloc memory.
    • Also internally uses get_symmetric_encrypt_authenticate_buffer() and get_symmetric_decrypt_authenticate_buffer()

c_crypto.h

• Add encryption mode, and IV size as #define macro.
• Move distribution_key_t and session_key_t to c_api.h.
• Update digest_message_SHA_256()'s description.
• Update description of newly added functions.

c_secure_comm.c

• Add send_state to check the state during the session key exchange steps.
• Change functions used only inside c_secure_comm.c to top, and make it static functions.
  • encrypt_and_sign(), serialize_session_key_req_with_distribution_key(), check_validity(), parse_distribution_key(), parse_session_key_response()
  • update_enc_mode_and_hmac_mode_to_session_key() is newly added.
• save_distribution_key(), parse_session_key() : Remove unused inputs, better type casting.
• Change the write() functions to write_to_socket()
• Match inputs for the changed inputs of symmetric_encrypt_authenticate() symmetric_decrypt_authenticate().
  • Functions changed: parse_handshake_1(), check_handshake_2_send_handshake_3(), decrypt_received_message(), check_handshake1_send_handshake2()
• send_SECURE_COMM_message(): Moved from c_api.c to c_secure_comm.c. This is not a new function.
  • Minor logic change: Does not malloc memory anymore. The length of the message to send is already fixed, so we can estimate the memory size to be used after encryption. Thus, it does not malloc() memory anymore, using the get_expected_encrypted_total_length() function.
• print_received_message() : free() memory.
• send_session_key_req_via_TCP():
  • Add states, and state checking, for better state handling during the session key request steps.
  • Match changed inputs on functions.
  • Add free() for memory deallocation.
  • Add AUTH_ALERT handling, by case.
• Comment out send_session_key_req_via_UDP(), due to compiler warnings.
• Newly added functions:
  • update_enc_mode_and_hmac_mode_to_session_key(): Update the encryption mode and hmac_mode from the SST_ctx_t to the session key itself.
  • encrypt_or_decrypt_buf_with_session_key(): Encrypts or decrypts the buffer. This malloc()s memory.
  • encrypt_or_decrypt_buf_with_session_key_without_malloc(): This does not do malloc() internally. The caller must allocate the buffer before using this function.

c_secure_comm.h

• Add auth_alert_code as enum.
• Move SST_session_ctx_t session_key_list_t SST_ctx_t struct typedefs to c_api.h. This is done because when installing c_api.h as a header file, it should not #include other header files such as c_secure_comm.h. If it does, other header files should also be installed, when doing sudo make install.
• Remove INIT_SESSION_KEY_LIST macro function, replaced with init_empty_session_key_list() in c_api.c.
• Remove static functions from c_secure_comm.h, and move the descriptions to c_secure_comm.c.
• Add descriptions for the newly added functions: encrypt_or_decrypt_buf_with_session_key(), encrypt_or_decrypt_buf_with_session_key_without_malloc().
• send_SECURE_COMM_message(): Moved from c_api.h to c_secure_comm.h

ipfs.c

• No logic changes, only updating changed function's inputs.

ipfs.h

• Only include stdio.h.

entity_server.c

• error_exit() is not an API function. Define same function and use it.

entity_client.c

• Include headers.

c_api.c

• Newly added functions:

  • init_empty_session_key_list() : Replaces the macro INIT_SESSION_KEY_LIST
  • encrypt_buf_with_session_key() / decrypt_buf_with_session_key() / encrypt_buf_with_session_key_without_malloc() / decrypt_buf_with_session_key_without_malloc()
  • save_session_key_list() / load_session_key_list()
  • save_session_key_list_with_password() / load_session_key_list_with_password()
  • convert_skid_buf_to_int()
  • generate_random_nonce()

• init_SST() :

  • Added OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT, NULL);. This should be removed in the future, and should be added by the API user.
  • Avoid compiler warnings.

• get_session_key() : Better error checking.

• secure_connect_to_server() / secure_connect_to_server_with_socket():

  • Add API available for the user to call the connected client socket itself.
  • Better error handlings.

• get_session_key_by_ID(): Better error checking and memory handling.

• server_secure_comm_setup():

  • Better error handling and use read_from_socket() instead of read().
  • Update inputs for the updated internal functions.

• receive_thread(), receive_thread_read_one_each(), receive_message() :

  • Update inputs for updated internal functions.

• free_SST_ctx_t: Update free function.

c_api.h

• Add descriptions for new functions.
• Move all structs to c_api.h. This is done to make SST as a library, only including c_api.h. c_api.h should not include other header files, or else other header files must be installed when doing sudo make install.

All config files.

• Add test configs for testing CBC, CTR, GCM mode and hmac mode.

All ipfs related files

• No logic changes. Only updates on inputs of changed functions.