What is Protocol Buffer?
I hope the reader is familiar with JSON format. It's a widely used data format for communicating between two different programs. Many mobiles and web applications use JSON for data exchange. Because it's easy to use and human-readable. Many IoT applications are also using JSON for data exchange. But there is a good tool that can help to cut down the data packets size sends over the internet. It's called Protocol Buffers.
Nanopb is an ANSI-C library for encoding and decoding messages in Google's Protocol Buffers format with minimal requirements for RAM and code space. It is primarily suitable for 32-bit microcontrollers.
Protocol buffers, usually referred to as Protobuf, is a protocol developed by Google to allow serialization and deserialization of structured data. Google developed it with the goal to provide a better way, compared to XML and JSON, to make systems communicate. So they focused on making it simpler, smaller, faster, and more maintainable than XML and JSON.
Microcontrollers use Protocol Buffers in IoT devices. It decreases the packet size, and reduces delivery time.
How do Protocol Buffers works?
The developer writes the Data structure schema in a .proto file. Then that file will be passed to the python script. Python script will generate a C header and C source file for encoding and decoding the data structure just now the developer described in the .proto file. User needs to use these C files inside their project directory to encode or decode. for more information refer to https://github.com/nanopb/nanopb
Process Overview
How to Install protocol buffers and dependencies?
Below mentioned are the Dependencies:
- Python
- Python Protobuf-Library
- Protoc
- Once the Python is installed then type the below command
pip install --upgrade protobuf grpcio-toolsIf it's successfully installed then good to go.
- Clone the nanopb GitHub directory
git clone https://github.com/nanopb/nanopb.gitHow to use protocol buffers?
- Once the installation is completed create a new directory inside nanopb. I prefer location nanopb/examples.
ex: directory named "sample" is created and its path looks like -> nanopb/examples/sample.
- Create a file name sample.proto
- "sample" is the name taken for example but you can name it related to your project.
What is a <sample.proto> file is all about?
If you observe the basic C structure, It needs definition before we can use it in the main program. Just like that, .proto is a file where the developer writes the data structure. It almost looks like C but with a little different syntax.
Ex:
Step 1: Creating a structure.
Type this in the sample.proto file
syntax = "proto3";
message Example {
int32 lucky_number = 1;
}
Here,
This tells the Protocol Buffer compiler to use protocol version 3. If it's not specified then by default it takes proto2 version.
"message" is the keyword used in the protocol buffer to create a structured schema followed by the user-defined message name. Here "Example" is the user-defined name.
just like uint32_t in C, we can use int32 to create a 32-bit integer variable in our code to store some sensor data or any other use desired values. Number 1 is used to identify the fields by Protobuf and it should be unique for a structure. There are number of data types supported by Protobuf.
click here to check different types of Data Types supported in Protobuf
https://jpa.kapsi.fi/nanopb/docs/concepts.html#data-types
Step 2: Generating C files.
Type the below command to generate C files in the terminal under the examples/sample directory.
This is exactly the C structure.
Step 2: Adding Protobuf to your project.
To Use the protobuf in your project you need the below-mentioned files
pb_encode.h/c
pb_decode.h/c
pb_common.h/c
pb.h
sample.ph.h/c : <message> same as the name of the .proto file
Physically Add all these files into your project directory or include a path to your project build system. message.ph.h/c files are generated from the previous command and the remaining files can be found on the root directory of the protobuf GitHub directory.
Include these files in the header of the C file where messages needs to to encoded or decoded.
#include "sample.ph.h"
#include "pb_encode.h"
#include "pb_decode.h"
If you use only encoding on your code then include only pb_encode.h file.
Example of Encoding and Decoding:
Once the protobuf generates the message.ph.h/c files copy and paste all the file names mentioned in <adding protobuf to your example> section in the root directory or include the path to your build system.
Since many embedded systems are platform dependent we are going to check encoding and decoding in basic C which is same in many C based embedded developments.
Create a c file named main.c in the root directory where all of your files are pasted. You can used those files stand alone in any other directory and there is no dependency for that since it all independent C file. But all files should be present in the same directory or path should be included in build system. once the files are included the project contains below files.
pb_encode.h/c
pb_decode.h/c
pb_common.h/c
pb.h
sample.ph.h/c
main.c :( C file to implement Encode Decode logics)
| #include <stdio.h> | |
| #include "pb_encode.h" | |
| #include "pb_decode.h" | |
| #include "sample.pb.h" | |
| int main() | |
| { | |
| /* This is the buffer where we will store our message. */ | |
| uint8_t buffer[128]; | |
| size_t message_length; | |
| bool status; | |
| /* Encode our message */ | |
| { | |
| /* Allocate space on the stack to store the message data. | |
| * | |
| * Nanopb generates struct definitions for all the messages. | |
| * - check out the contents of sample.pb.h! | |
| * It is a good idea to always initialize your structures | |
| * so that you do not have garbage data from RAM in there. | |
| */ | |
| SampleMessage message = SampleMessage_init_zero; | |
| /* Create a stream that will write to our buffer. */ | |
| pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer)); | |
| /* Fill in the lucky number */ | |
| message.lucky_number = 13; | |
| /* Now we are ready to encode the message! */ | |
| status = pb_encode(&stream, SampleMessage_fields, &message); | |
| message_length = stream.bytes_written; | |
| /* Then just check for any errors.. */ | |
| if (!status) | |
| { | |
| printf("Encoding failed: %s\n", PB_GET_ERROR(&stream)); | |
| return 1; | |
| } | |
| } | |
| /* Now we could transmit the message over network, store it in a file or | |
| * wrap it to a pigeon's leg. | |
| */ | |
| /* But because we are lazy, we will just decode it immediately. */ | |
| { | |
| /* Allocate space for the decoded message. */ | |
| SampleMessage message = SampleMessage_init_zero; | |
| /* Create a stream that reads from the buffer. */ | |
| pb_istream_t stream = pb_istream_from_buffer(buffer, message_length); | |
| /* Now we are ready to decode the message. */ | |
| status = pb_decode(&stream, SampleMessage_fields, &message); | |
| /* Check for errors... */ | |
| if (!status) | |
| { | |
| printf("Decoding failed: %s\n", PB_GET_ERROR(&stream)); | |
| return 1; | |
| } | |
| /* Print the data contained in the message. */ | |
| printf("Your lucky number was %d!\n", (int)message.lucky_number); | |
| } | |
| return 0; | |
| } |
Copy and pase the above code in main.c file. Comments sections in the code explains code very well.
Once the files are ready now lets compile the code and run.
Run in the files workspace directory :
gcc */c -o main
this will compile and generate executable file.
To run the code type below command
./main
output will be
Your lucky number was 13!
