Use the in_http plug-in in Fluentd to
I would like to send the json file and register it with elasticsearch.
Currently, I am trying to handle the following json files:
{
"user_id": 0,
"time"—0,
"data": [
{
"data0": 0.0,
"data1"—0.1
},
{
"data0": 0.0,
"data1"—0.2
}
]
}
I was able to confirm that one piece of data was successfully registered.
"However, I would like to register each element of the ""data"" array as a different id, so
"
To make bulkAPI available,
{"index":{"_index":<index_name>, "_type":<type_name>, "_id":<id>}}
{ "user_id":0, "time":0, "data":{"data0":0.0, "data1":0.1}
{ "index": {"_index": <index_name>, "_type": <type_name>, "_id": <id>}
{ "user_id":0, "time":0, "data":{"data0":0.0, "data1":0.2}}
We are considering converting it to the form shown in and throwing it into elasticsearch.
About its implementation:
·How to pass index_name/type_name to Fluentd via http POST
·How to convert to bulkAPI format on fluidd
I'm in trouble because I don't know.
For instructions on registering in bulkAPI format with fluidd-elasticsearch, see
If anyone knows a good way, please let me know.
Thank you.
The various versions are as follows.
OS:CentOS Linux release 7.5.1804(Core)
td-agent:2.3.6
elasticsearch:5.5.3
Use fluent-plguin-elasticsearch.
In_http is used differently because it allows Fluentd to accept HTTP requests.
© 2024 OneMinuteCode. All rights reserved.