MinIO as Storage

1. What is MinIO?

• MinIO is a high-performance, distributed object storage system that is compatible with the Amazon S3 API.
• It's often used in self-hosted environments to provide object storage for applications like Loki.

2. How Loki Uses MinIO

• Loki stores its log data (chunks) in MinIO as objects.
• Metadata (indexes) may also be stored in MinIO or another supported backend, depending on your configuration.

3. Loki-MinIO Configuration

The Loki setup with MinIO typically works as follows:

• MinIO Pod: The pod you see (loki-minio-0) runs a MinIO instance, which serves as the object storage.
• Loki Configuration: Loki is configured to use this MinIO instance as its storage backend.

The configuration might look like this in loki-config.yaml:

storage_config:
  aws:
    s3: http://loki-minio:9000
    bucketnames: loki
    access_key_id: <your-minio-access-key>
    secret_access_key: <your-minio-secret-key>

4. How Data Flows in This Setup

1. Log Collection:
   • Logs from Kubernetes pods are collected by agents like Promtail or Fluent Bit.
2. Log Ingestion:
   • Logs are ingested into Loki, which splits them into chunks and indexes.
3. Storage in MinIO:
   • Chunks (compressed log data) are stored as objects in MinIO.
   • Indexes are stored either in MinIO or another indexing backend like BoltDB or DynamoDB.
4. Querying via Grafana:
   • When you query logs in Grafana, it asks Loki.
   • Loki retrieves the relevant log chunks and metadata from MinIO to fulfill the query.

5. Verifying MinIO Storage

You can check the logs and storage directly in MinIO:

1. Access MinIO UI (if enabled):
   • MinIO typically runs a web interface on port 9000.
   • Access it via http://:9000 or through a Kubernetes service.
2. List Stored Chunks:
   • Once logged in, you’ll see the loki bucket.
   • Inside, you’ll find folders corresponding to different log streams.

6. Useful Commands

• Check Loki-MinIO Connection: Inspect Loki’s logs to see if it’s writing to MinIO successfully:

kubectl logs <loki-pod> -n <namespace>

• Check MinIO Logs:

kubectl logs loki-minio-0 -n <namespace>