Accessing Kafka

Read(142) Label: kafkacli,

1. Download the following third-party jars for connecting to the Kafka server from Kafka environment and put them in esProc external library folder. The directory containing files of this external library is: installation directory\esProc\extlib\KafkaCli. The Raqsoft core jar for this external library is KafkaCli.jar.

kafka_2.11-1.1.0.jar

log4j-1.2.17.jar

kafka-clients-1.1.0.jar

slf4j-api-1.7.25.jar

slf4j-log4j12-1.7.25.jar

Note: The third-party jars are provided within the package and users can choose appropriate ones for specific scenarios.

 

2. A JRE version 1.7 or above is required. The embedded JRE version in esProc is JRE1.6. Users need to install a higher version and configure java_home in the config.txt under installation directory\esProc\bin. If a JDK version 1.7 or above has been chosen when installing esProc, just ignore this step.

 

3. Users can manually change the size of memory if the default size isn’t large enough for needs. Two ways to manage memory under Windows are available: Change the memory settings in config.txt when starting esProc through the executable file; and in the .bat file when starting the application through the batch file. Under Linux, change the memory size in the .sh file.

Below is the method of changing memory settings in config.txt under Windows:

java_home=C:\ProgramFiles\Java\JDK1.7.0_11;esproc_port=48773;jvm_args=-Xms256m-XX:PermSize=256M-XX:MaxPermSize=512M-Xmx9783m-Duser.language=zh

 

4. esProc provides four library functions – kafka_subscribe(), kafka_close(), kafka_poll(), and kafka_commit() - to access the Kafka. Look up them inHelp-Function referenceto find the uses.