how to tokenise a string in c
problem
You wish to extract tokens from a string using a delimeter. For example, a string that contains a line of code/ instruction and you wish to break it apart to further evaluate it.
SOLUTION
In C, we can find the strtok() function that helps us to break a given string into tokens using a delimeter/ separator character (e.g. a comma, tab). Below we will see an example with comma:
#include <stdio.h>
#include <string.h>
int main() {
char instruction[100] = "add $v1,$zero,$zero";
//First token
char *token = strtok(instruction, ",");
//extract tokens
while(token != NULL) {
printf("%s\n", token); //printing each token
token = strtok(NULL, ",");
}
return 0;
}
At line 6, we have the instruction that we wish to break into tokens using comma as separator.
At line 9, we get the first token. Then at line 12 the loop is used to extract the remaining tokens and print each one.
running
Compile:
gcc token.c
Execute:
./a.out
output
The instruction is split into 3 tokens as expected.
- add $v1
- $zero
- $zero
conclusion
In this post we saw how to use the strtok() function in C to split a given string into tokens using comma as a delimeter – separator.