Skip to content

cloudflare_logpush.md

Latest commit

 

History

History
84 lines (68 loc) · 4.23 KB

cloudflare_logpush.md

File metadata and controls

84 lines (68 loc) · 4.23 KB
title Send Cloudflare Logpush Logs to Observability Pipelines
description Learn how to send Cloudflare Logpush logs to Observability Pipelines
disable_toc false
products
name icon url
Logs
logs
/observability_pipelines/configuration/?tab=logs#pipeline-types

{{< product-availability >}}

Overview

This document goes over how to send Cloudflare Logpush logs to Observability Pipelines using the HTTP Server source.

Prerequisites

The following are required to send Cloudflare Logpush logs to Observability Pipelines:

  • A Cloudflare account with Logpush enabled.
  • A server or a server pool, fronted by a load balancer, that runs the Observability Pipelines Worker and allows traffic from Cloudflare's CIDR blocks or the public internet.
  • A DNS entry that points to your Workers' load balancer or OPW server.
  • An SSL/TLS certificate for your domain. Cloudflare requires an HTTPS endpoint and does not accept HTTP.
    • Note: You cannot use Cloudflare origin certificates because they are not publicly trusted.
  • If you are using a firewall in front of your Worker or load balancer, add Cloudflare's IP addresses to your allowlist.

Set up a pipeline

Set up the pipeline components

  1. Navigate to Observability Pipelines.
  2. Select a log template to create a pipeline.
  3. Select the HTTP Server source.
  4. If you are using Secrets Management, enter the identifier for the HTTP/S Server address key. See Set secrets for the defaults used.
  5. Select your authorization strategy. If you selected Plain:
    • Enter the identifiers for the HTTP/S Server username and password. See Set secrets for the defaults used.
  6. In the Decoding dropdown menu, select Bytes. {{% observability_pipelines/tls_settings %}}
  7. Copy your certificates into the configuration directory: 
    # Create the configuration directory
sudo mkdir -p /var/lib/observability-pipelines-worker/config

# Copy your certificates
sudo cp /path/to/your/<your-cert-file> /var/lib/observability-pipelines-worker/config/<your-cert-file>
sudo cp /path/to/your/<your-cert-file> /var/lib/observability-pipelines-worker/config/<your-cert-file>
  8. After you set up your destinations and processors, click Next: Install.

Install the Worker

  1. On the Install page, select your platform in the dropdown menu.
  2. Follow the instructions on the page to install the Worker based on your platform. See Install the Worker for details.
  3. After installing the Worker, change ownership of the certificates so the Observability Pipelines Worker can read them: 
    # Change ownership so the Worker can read the certificates
sudo chgrp observability-pipelines-worker /var/lib/observability-pipelines-worker/config/<your-cert-file>
sudo chmod 640 /var/lib/observability-pipelines-worker/config/<your-cert-file>
sudo chgrp observability-pipelines-worker /var/lib/observability-pipelines-worker/config/<your-cert-file>
sudo chmod 640 /var/lib/observability-pipelines-worker/config/<your-cert-file>
  4. Deploy the configuration from the Observability Pipelines UI.
  5. Test your endpoint using curl: 
    curl -X POST https://your-domain.com \
  -u username:password \
  -H "Content-Type: application/json" \
  -d '{"message":"test log from curl","source":"curl","service":"cloudflare-logpush"}'

Set up Cloudflare Logpush

Follow the Cloudflare Logpush HTTP destination documentation to set up Logpush to send logs to an HTTP endpoint.

  • For the HTTP endpoint: If you use HTTP Basic authorization, the authorization headers in the URL need to be base64 encoded: https://subdomain.your-domain.com?header_Authorization=Basic%20<base64-encoded-credentials>

After your Logpush job has been successfully created, you can view your Cloudflare Logpush logs in Datadog Log Explorer.