TechSNAP - Episodes Tagged with “Monitoring”

428: RAID Reality Check

Jupiter Broadcasting — Fri, 01 May 2020 00:15:00 -0700

We dive deep into the world of RAID, and discuss how to choose the right topology to optimize performance and resilience.

Plus Cloudflare steps up its campaign to secure BGP, and why you might want to trade in cron for systemd timers.

Links:

422: Multipath Musings

Jupiter Broadcasting — Fri, 07 Feb 2020 00:15:00 -0800

We take a look at a few exciting features coming to Linux kernel 5.6, including the first steps to multipath TCP.

Plus the latest Intel speculative execution vulnerability, and Microsoft's troubled history with certificate renewal.

Links:

407: Old School Outages

Jupiter Broadcasting — Wed, 10 Jul 2019 22:00:00 -0700

Jim shares his Nagios tips and Wes chimes in with some modern tools as we chat monitoring in the wake of some high-profile outages.

Plus we turn our eye to hardware and get excited about the latest Ryzen line from AMD.

Links:

Third parties confirm AMD’s outstanding Ryzen 3000 numbers | Ars Technica — AMD debuted its new Ryzen 3000 desktop CPU line a few weeks ago at E3, and it looked fantastic. For the first time in 20 years, it looked like AMD could go head to head with Intel's desktop CPU line-up across the board. The question: would independent, third-party testing back up AMD's assertions?
The Internet broke today: Facebook, Verizon, and more see major outages | Ars Technica — Last week, Verizon caused a major BGP misroute that took large chunks of the Internet, including CDN company Cloudflare, partially down for a day. This week, the rest of the Internet has apparently asked Verizon to hold its beer.
It was a really bad month for the internet | TechCrunch — In the past month there were several major internet outages affecting millions of users across the world. Sites buckled, services broke, images wouldn’t load, direct messages ground to a halt and calendars and email were unavailable for hours at a time.
Cloudflare outage caused by bad software deploy (updated) — For about 30 minutes today, visitors to Cloudflare sites received 502 errors caused by a massive spike in CPU utilization on our network. This CPU spike was caused by a bad software deploy that was rolled back.
How Verizon and a BGP Optimizer Knocked Large Parts of the Internet Offline Today — Today at 10:30UTC, the Internet had a small heart attack. A small company in Northern Pennsylvania became a preferred path of many Internet routes through Verizon (AS701), a major Internet transit provider.
Getting started | Prometheus — This guide is a "Hello World"-style tutorial which shows how to install, configure, and use Prometheus in a simple example setup.
prometheus/node_exporter — Prometheus exporter for hardware and OS metrics exposed by *NIX kernels, written in Go with pluggable metric collectors.
Using netdata with Prometheus — Prometheus is a distributed monitoring system which offers a very simple setup along with a robust data model. Recently netdata added support for Prometheus.
prometheus/nagios_plugins — Nagios plugin for alerting on prometheus query results.
RobustPerception/nrpe_exporter — The NRPE exporter exposes metrics on commands sent to a running NRPE daemon.
m-lab/prometheus-nagios-exporter — The Prometheus Nagios exporter reads status and performance data from nagios plugins via the MK Livestatus Nagios plugin and publishes this in a form that can be scrapped by Prometheus.
Comparison to alternatives | Prometheus — Prometheus is a full monitoring and trending system that includes built-in and active scraping, storing, querying, graphing, and alerting based on time series data.
Quality server monitoring solution using NetData/Prometheus/Grafana — I’m going to quickly show you how to install both netdata and Prometheus on the client and server. We can then use grafana pointed at Prometheus to obtain long-term metrics netdata offers.
Monitoring stack by using Grafana + Prometheus + Netdata — This monitoring stack you can monitoring in real-time by Netdata and see the history by using Grafana.
Monitoring Agent · NCPA — New to NCPA? See some of the awesome features present in the Web GUI and API, available on any operating system.
Nagios 101: Understanding the Fundamentals - Nagios
Nagios Documentation

397: Quality Tools

Jupiter Broadcasting — Thu, 14 Feb 2019 21:00:00 -0800

Join Jim and Wes as they battle bufferbloat, latency spikes, and network hogs with some of their favorite tools for traffic shaping, firewalling, and QoS.

Plus the importance of sane defaults and why netdata belongs on every system.

Links:

Why you want QoS - Netdata Documentation — One of the features the Linux kernel has, but it is rarely used, is its ability to apply QoS on traffic. Even most interesting is that it can apply QoS to both inbound and outbound traffic.
FireQOS Wiki — FireQOS is a helper to assist you configure traffic shaping on Linux.
FireHOL - Linux firewalling and traffic shaping for humans — FireHOL is a language (and a program to run it) which builds secure, stateful firewalls from easy to understand, human-readable configurations. The configurations stay readable even for very complex setups.
tc(8) man page — Traffic Control consists of the following: SHAPING When traffic is shaped, its rate of transmission is under control. Shaping may be more than lowering the available bandwidth - it is also used to smooth out bursts in traffic for better network behaviour. Shaping occurs on egress. SCHEDULING By scheduling the transmission of packets it is possible to improve interactivity for traffic that needs it while still guaranteeing bandwidth to bulk transfers. Reordering is also called prioritizing, and happens only on egress. POLICING Where shaping deals with transmission of traffic, policing pertains to traffic arriving. Policing thus occurs on ingress. DROPPING Traffic exceeding a set bandwidth may also be dropped forthwith, both on ingress and on egress.
Overview of Traffic Control Concepts — Traffic control is the name given to the sets of queuing systems and mechanisms by which packets are received and transmitted on a router. This includes deciding which (and whether) packets to accept at what rate on the input of an interface and determining which packets to transmit in what order at what rate on the output of an interface.
Advanced traffic control - ArchWiki
Journey to the Center of the Linux Kernel: Traffic Control, Shaping and QoS — This document describes the Traffic Control subsystem of the Linux Kernel in depth, algorithm by algorithm, and shows how it can be used to manage the outgoing traffic of a Linux system.
Netdata Real-time performance monitoring, done right! — Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
Add more charts to netdata — To collect non-system metrics, netdata supports a plugin architecture.

Episode 388: The One About eBPF

Jupiter Broadcasting — Thu, 25 Oct 2018 15:00:00 -0700

We explain what eBPF is, how it works, and its proud BSD production legacy.

eBPF is a technology that you’re going to be hearing more and more about. It powers low-overhead custom analysis tools, handles network security in a containerized world, and powers tools you use every day.

Links:

Chris Goes to MeetBSD
Linus Torvalds talks about coming back to work on Linux | ZDNet — BPF has actually been really useful, and the real power of it is how it allows people to do specialized code that isn't enabled until asked for.
The Kernel Report - Jonathan Corbet
BPF - the forgotten bytecode — All this changed in 1993 when Steven McCanne and Van Jacobson published the paper introducing a better way of filtering packets in the kernel, they called it "The BSD Packet Filter" (BPF)
The BSD Packet Filter
eBPF: Past, Present, and Future — The Extended Berkeley Packet Filter, or eBPF, has rapidly been adopted into a number of Linux kernel systems since its introduction into the Linux kernel in late 2014. Understanding eBPF, however, can be difficult as many try to explain it via a use of eBPF as opposed to its design. Indeed eBPF's name indicates that it is for packet filtering even though it now has uses which have nothing to do with networking.
Using eBPF in Kubernetes — Cilium is a networking project that makes heavy use of eBPF superpowers to route and filter network traffic for container-based systems. By using eBPF, Cilium can dynamically generate and apply rules—even at the device level with XDP—without making changes to the Linux kernel itself
Why is the kernel community replacing iptables with BPF? — The Linux kernel community recently announced bpfilter, which will replace the long-standing in-kernel implementation of iptables with high-performance network filtering powered by Linux BPF, all while guaranteeing a non-disruptive transition for Linux users.
bpftrace (DTrace 2.0) for Linux 2018 — Created by Alastair Robertson, bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. It's shaping up to be a DTrace version 2.0: more capable, and built from the ground up for the modern era of the eBPF virtual machine.
The bpftrace One-Liner Tutorial
BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more — BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools and examples.
Linux eBPF Tracing Tools — This page shows examples of performance analysis tools using enhancements to BPF (Berkeley Packet Filter) which were added to the Linux 4.x series kernels, allowing BPF to do much more than just filtering packets. These enhancements allow custom analysis programs to be executed on Linux dynamic tracing, static tracing, and profiling events.
eBPF Vulnerability (CVE-2017-16995): When the Doorman Becomes the Backdoor
Ultimate Plumber — Ultimate Plumber is a tool for writing Linux pipes with instant live preview
BSD Now 073: Pipe Dreams — Interview w/ David Maxwell about Pipecut, text processing, and commandline wizardry.