Expanding Your SIEM Stack: How to Ingest Webhook-Only Logs with Shuffle and CoPilot

When building a robust SIEM stack, one of the most fundamental goals is comprehensive log ingestion. Traditionally, this means pulling logs from endpoints using agents like Wazuh, or forwarding syslog data from network infrastructure to platforms like Graylog. But what happens when you’re dealing with applications that don’t support either?

These are the “pesky” apps — tools that don’t write to a local file, don’t expose a usable API, and yet generate events you care deeply about.

In this post, I’ll walk you through a workflow that enables you to ingest webhook-only logs into your internal SIEM using a powerful combination of:

• Shuffle (SOAR platform)
• CoPilot (our internal case management + log shipper)
• Graylog (log storage, normalization, and multi-tenancy)

The Challenge: External Webhooks, Internal SIEM

Some services — Cloudflare being one example — can’t forward logs via syslog or agent. Instead, they rely on outbound webhook notifications. That means you need a publicly accessible listener. But hosting a public webhook inside your internal network introduces risks and complexity: firewall changes, public exposure, and network architecture adjustments.

So, how do you securely bridge the external app and your internal SIEM?

The Solution: Using Shuffle + CoPilot to Ingest Webhook Events

Here’s the architectural breakdown:

1. Shuffle Cloud receives the webhook (publicly accessible).
2. Shuffle Hybrid Worker forwards the data internally.
3. CoPilot processes and routes the data to Graylog.
4. Graylog stores it in a customer-specific index based on metadata.

This gives you the best of both worlds:

• Public webhook endpoint (via Shuffle Cloud)
• Private log ingestion pipeline (via Shuffle Hybrid Runtime + CoPilot)

Step-by-Step Workflow Overview

1. Set Up Your Webhook in Shuffle

Within Shuffle, create a new workflow and add the Webhook app as the starting point. This will generate a public URL that you can test with a simple curl request.

curl -X POST <webhook_url> -H "Content-Type: application/json" -d '{"message":"test"}'

2. Add Metadata for Multi-Tenancy

Using Shuffle’s Set JSON Key app, append two key pieces of metadata to the webhook payload:

• integration: e.g., “cloudflare”
• customer_code: e.g., “lab”

These are crucial for routing and indexing downstream.

3. POST to CoPilot Using the HTTP App

The enriched payload is then POSTed to your internal CoPilot endpoint using the HTTP app inside Shuffle. Be sure to set the app’s runtime location to your hybrid worker (the one inside your network).

http://YOUR_COPILOT_IP:5000/api/graylog/receiver

4.Ingest into Graylog

CoPilot receives the event and forwards it to your configured Graylog input. From here, Graylog will write the event — initially in the default index.

5.Set Up Custom Graylog Index and Stream

To organize events by source and customer, create:

• An Index Set named cloudflare-lab
• A Stream that matches on integration: cloudflare and customer_code: lab

This ensures that Cloudflare logs for the “lab” tenant are isolated, searchable, and retained according to your policies.

Why This Matters

Not all apps are built for enterprise logging. Some lack agents, APIs, or syslog support — but they still produce valuable security data. By implementing this workflow, you:

• Avoid risky firewall changes
• Support webhook-only apps
• Preserve multi-tenancy and structured routing
• Keep your SIEM extensible, secure, and adaptable

This setup also makes it easier to onboard new tools in the future that only offer webhook outputs. Instead of rewriting your stack, you simply create a new Shuffle workflow, define integration metadata, and let the pipeline do the rest.

Final Thoughts

This isn’t the only way to solve webhook-to-SIEM challenges, but it’s an elegant and scalable one. By combining Shuffle’s cloud/hybrid capabilities with CoPilot’s ingestion and Graylog’s indexing, we’ve built a flexible bridge between cloud apps and internal security operations.

Need Help?

The functionality discussed in this post, and so much more, are available via the SOCFortress platform. Let SOCFortress help you and your team keep your infrastructure secure.

Website: https://www.socfortress.co/

Contact Us: https://www.socfortress.co/contact_form.html