davidbelhamou/fluentd — reverse-engineered prompt

Build me a small working proof of concept that replaces an old Python log shipper with Fluentd.

I want one central Fluentd service that reads rotated JSON log files from a shared folder like an NFS mount, remembers where it left off so it doesn’t resend old lines, adds the source file name to each record, and sends the logs into Elasticsearch data streams. It should route different service folders or log types into the right data stream, with a fallback stream for anything unknown.

Please include sample Python services that generate rotating log files so I can test it locally, plus a cleanup job that removes log files after Fluentd has safely read them. Add Docker and Docker Compose so I can run the whole thing on my machine, and keep the Fluentd config easy to move into an OpenShift ConfigMap later.

Use sensible Elasticsearch templates and an ILM setup script. Look up current Fluentd and Elasticsearch docs online if needed.