What is Filebeat?
The one-paragraph definition
Filebeat is a lightweight open-source log shipper by Elastic (part of the Beats family). It tails log files on a host, tracks read offsets, and streams events to a destination — most often Elasticsearch, Logstash, Kafka, or a managed log platform.
Written in Go, Filebeat is distributed as a single ~30 MB binary with minimal resource overhead (typically 20–60 MB RAM, under 1% CPU on a busy host). Safe to install on every production node.
Filebeat is part of the ELK (Elasticsearch, Logstash, Kibana) stack but can ship logs to any supported output, not just Elastic. OpenSearch, Kafka, Redis, and HTTP endpoints all work.
In 2026 Filebeat still dominates ELK deployments; for Kubernetes-first teams, Fluent Bit is a smaller alternative. Vector is a newer Rust-based option that is gaining share.
Filebeat architecture
How Filebeat actually works
Harvester: a Filebeat component spawns one harvester per file being monitored. The harvester tails the file, reading new lines as they are written.
Input: a group of files (e.g., /var/log/nginx/*.log) that Filebeat should watch. Each input can have its own fields, tags, and processors.
Registry: Filebeat writes read offsets to a local registry file. If Filebeat restarts, it resumes from the last read position — no duplicate events, no gaps.
Spooler + output: events are batched in memory and shipped to the configured output (Elasticsearch, Logstash, Kafka, etc.) with retry on failure.
Processors: lightweight transformations before shipping (add_host_metadata, drop_fields, parse_timestamp). More complex parsing happens downstream in Logstash or Elasticsearch ingest pipelines.
Basic Filebeat setup
From "zero" to "logs flowing"
Install Filebeat on each host: on Debian/Ubuntu, apt install filebeat; on RHEL/CentOS, yum install filebeat; on Docker/Kubernetes, use the official Elastic image.
Configure /etc/filebeat/filebeat.yml with inputs (which files to tail) and output (where to send logs). The simplest example: input is /var/log/*.log, output is Elasticsearch at an URL.
Enable Filebeat modules for common log formats: filebeat modules enable nginx mysql kubernetes. Modules include preconfigured inputs plus Elasticsearch ingest pipelines for parsing.
Start the service: systemctl enable filebeat && systemctl start filebeat. Check status with filebeat test output to verify connectivity to your destination.
In Kubernetes, deploy Filebeat as a DaemonSet (one pod per node) with autodiscover enabled — Filebeat auto-detects pod labels and attaches them as fields.
Filebeat modules
Preconfigured parsers for 60+ sources
A Filebeat module bundles: an input configuration (where to find the logs), a parser (Elasticsearch ingest pipeline), and Kibana dashboards.
Common modules: nginx, apache, mysql, postgresql, kafka, kubernetes, aws, system (syslog), auditd, redis, haproxy, iis, mongodb, nats, osquery, suricata, zeek.
Modules are a time-saver: instead of writing a grok pattern for Nginx logs, enable the Nginx module and the fields (http.method, http.status_code, source.ip, user_agent.*) populate automatically.
List available modules: filebeat modules list. Enable one: filebeat modules enable nginx. Configure its paths in /etc/filebeat/modules.d/nginx.yml.
Filebeat vs Logstash vs Fluent Bit vs Vector
Which log shipper to use
Filebeat: lightweight Go-based shipper. Minimal transforms. Best for: ship logs with low overhead, pair with downstream parsing (Logstash, ingest pipelines).
Logstash: Java-based ETL. Heavy (500 MB+ RAM) with a rich filter DSL (grok, mutate, geoip). Best for: central tier doing complex parsing and routing.
Fluent Bit: C-based shipper, even smaller than Filebeat (3 MB binary, <10 MB RAM). Includes basic parsing. Best for: Kubernetes environments where resource is critical.
Vector (by Datadog, open source): Rust-based, can replace Filebeat + Logstash with one tool. Best for: performance-sensitive deployments with transforms.
For managed destinations like Atatus, any of these works — Atatus accepts Filebeat, Fluent Bit, Vector, and OTel directly, so the shipper is your choice.
When to use Filebeat (and when not to)
Practical selection advice
Use Filebeat when: your logs land in files, you want minimal overhead per host, and your destination is Elasticsearch, OpenSearch, or any ELK-adjacent stack.
Use Fluent Bit when: you are Kubernetes-native and resource overhead matters more than familiarity with Elastic tooling.
Use Vector when: you want to consolidate Filebeat + Logstash into one agent with modern config and better performance.
Use OpenTelemetry Collector when: you are shipping telemetry (traces + metrics + logs) and want vendor-neutral instrumentation.
Don't use Filebeat when: your logs don't land in files (containers writing to stdout and journald is fine; proprietary binary logs are harder).
Key Takeaways
- Filebeat = lightweight Go-based log shipper from Elastic. Tails files, tracks offsets, ships to destinations.
- Architecture: harvesters tail files, input groups them, registry tracks offsets, output ships them.
- Modules provide preconfigured parsers for 60+ sources (nginx, mysql, kubernetes, aws, syslog).
- Resource overhead: ~30 MB binary, 20–60 MB RAM, <1% CPU — safe on every host.
- Alternatives: Logstash (heavy, rich transforms), Fluent Bit (smaller, K8s-first), Vector (Rust, consolidation).
- Pair with managed log platforms (Atatus, OpenSearch Service) to avoid running the storage tier yourself.