CloudPro

Infamous DevOps roadmapCloudPro #66: Grafana Soars Past $250M ARRWhat changed in the way you code for 2024? What has happened in the tech world in the last months? Take this shorter version of the Developer Nation survey, learn about new tools, influence the future of development and share your insights with the world!TAKE THE SURVEY⭐MasterclassInfamous DevOps roadmapKubernetes Open Source Limits & Requests Configuration OptimizationA guide to modern Kubernetes network policiesUsing Python Virtual Environments in DockerHow to terminate Go programs elegantly – a guide to graceful shutdowns🔍Secret KnowledgeHow Meta Enforces Purpose Limitation at ScaleWhy I Use Nim Instead of Python for Data ProcessingConvert OpenTelemetry Traces to Metrics using SpanMetrics ConnectorWhat happens when bucket.grantRead() in AWS CDKPreventing the Risk of Request Collapsing in Web Caching⚡TechwaveGrafana Labs Soars Past $250M ARR and 5,000 Customers, Completes $270M funding round, and Named a Leader in the Gartner Magic Quadrant for Observability PlatformsCockroachDB retires its free "Core" versionOpenMetrics is Archived, Merged into PrometheusAnnouncing Storage Browser for Amazon S3 for your web applications (alpha release)Juniper jumps into Wi-Fi 7 with enterprise switches, access points🛠️HackhubKardinal: lightest-weight way to spin up dev and test environments in KubernetesKubeblocks: control plane software that runs and manages databases, message queues on K8s.Flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solutionKubecolor: Colorize your kubectl outputAWS-mine: AWS honey token manager💡Recommended Learning: Continuous Integration Mastery with JenkinsCheers,Shreyans SinghEditor-in-ChiefIntroducing A Market-Changing Approach to Mobile App Protection by GuardsquareMobile applications face constant, evolving threats.To address these challenges, Guardsquare is proud to announce the launch of our innovative guided configuration approach to mobile app protection.Learn More⭐MasterClass: Tutorials & GuidesInfamous DevOps roadmapThis roadmap provides community-driven guides, resources, and roadmaps to help developers grow in their careers, focusing on different fields like DevOps, backend development, and various programming languages. It offers step-by-step instructions for learning new skills, tracking progress, and staying updated with industry best practices.Kubernetes Open Source Limits & Requests Configuration OptimizationThis article provides a step-by-step guide on using Kexa, an open-source tool for optimizing Kubernetes resource limits and requests through monitoring and alerting, with Grafana for visualization. It explains how to install Kexa using Helm, set up necessary credentials, connect it to databases like Postgres or MySQL, and configure rules to monitor CPU and memory consumption. It then walks through the setup of a Grafana dashboard to display and optimize pod performance.A guide to modern Kubernetes network policiesIn Kubernetes, network policies are rules that control traffic flow between pods in a cluster. They define which traffic is allowed to enter (ingress), exit (egress), or move between pods, helping secure communication within the cluster. These policies fall into two categories based on the OSI model: Layer 4 (L4) policies, which control traffic using IP addresses and ports, and Layer 7 (L7) policies, which offer finer control at the application level (e.g., HTTP routes). By combining both, Kubernetes can implement robust, zero-trust security models.Using Python Virtual Environments in DockerThe author explains that despite the trend of simplifying Python Docker workflows by avoiding virtual environments, they continue using them for several key reasons. Virtual environments provide predictability, a well-defined structure, and consistency across projects, which simplifies communication and management in team environments. By isolating the Python environment, it helps prevent complex import issues and makes the codebase more reliable and easier to debug.How to terminate Go programs elegantly – a guide to graceful shutdownsBy handling termination signals like SIGTERM, Go applications can stop accepting new requests while allowing in-flight processes to finish, utilizing tools such as `signal.NotifyContext` and `sync.WaitGroup` to manage concurrency. This approach helps maintain data integrity and smooth operations during shutdowns, particularly in orchestrated environments where unexpected terminations can otherwise lead to issues.🔍Secret Knowledge: Learning ResourcesHow Meta Enforces Purpose Limitation at ScaleMeta enforces purpose limitation at scale using its Privacy Aware Infrastructure (PAI) through technologies like Policy Zones. Policy Zones ensure that data is processed only for its intended purposes by labeling and tracking data assets across systems. It integrates real-time checks during data flow, preventing unauthorized uses by monitoring the movement and processing of data in different environments like function-based or batch-processing systems. This approach provides granular control over data use while scaling across Meta’s complex infrastructure.Why I Use Nim Instead of Python for Data ProcessingThe author chooses Nim over Python for data processing because it offers the simplicity of Python with the speed of C, making it ideal for handling large datasets without complex optimization. In a comparison of processing a 150 MB genome file, Nim significantly outperforms Python, running 30 times faster with nearly identical code. While Nim requires a few syntax changes, such as using `var` for variables and `echo` for output, its faster compilation and execution make it a powerful alternative for tasks like analyzing DNA sequences.Convert OpenTelemetry Traces to Metrics using SpanMetrics ConnectorThe SpanMetrics Connector in OpenTelemetry allows you to convert trace data into actionable metrics, addressing the lack of native metrics support in some languages. It works by aggregating key metrics like request counts, errors, and durations (R.E.D metrics) from trace spans. By configuring it in the OpenTelemetry Collector, you can generate useful performance insights without adding extra instrumentation for metrics.What happens when bucket.grantRead() in AWS CDKWhen you call `bucket.grantRead()` in AWS CDK, it grants read permissions to an IAM role or user by either updating identity-based policies (attached to the IAM principal) or resource-based policies (attached to the S3 bucket). If the IAM role was created within the same CDK stack, identity-based policies are updated. However, if the role or bucket is just a reference (using interfaces like `IRole` or `IBucket`), CDK cannot modify existing policies, and the grant may not work.Preventing the Risk of Request Collapsing in Web CachingRequest collapsing is a caching feature where multiple identical requests for the same resource are combined, so only one is sent to the origin server to reduce load. However, this can cause security issues when dealing with sensitive data, as the response to the first request might be mistakenly sent to other users who made the same request. Even if a server uses `Cache-Control: no-cache`, request collapsing may still send cached responses to multiple users. To prevent this, it's crucial to use strict cache policies, such as disabling caching for certain patterns and configuring both the cache and origin server to avoid caching sensitive data.⚡TechWave: Cloud News & AnalysisGrafana Labs Soars Past $250M ARR and 5,000 Customers, Completes $270M Primary and Secondary Transaction, and Named a Leader in the Gartner® Magic Quadrant™ for Observability PlatformsCockroachDB retires its free "Core" versionCockroachDB is evolving its self-hosted offering by retiring the free "Core" version and consolidating all users into a single "Enterprise" version that provides full access to its advanced features. This change, starting with version 24.3, ensures that individuals, students, and small businesses (under $10M annual revenue) can still use CockroachDB Enterprise for free with community support, while larger businesses will need a paid license.OpenMetrics is Archived, Merged into PrometheusThe OpenMetrics project, originally created to spin off Prometheus' metrics format into an independent specification, has been archived and merged back into Prometheus as of July 2024. While OpenMetrics aimed to become a universal format for exporting metrics, it struggled to gain adoption outside the Prometheus ecosystem, where Prometheus had already become the de facto standard for cloud-native observability.Announcing Storage Browser for Amazon S3 for your web applications (alpha release)Amazon S3 has released an alpha version of Storage Browser for S3, an open-source component that lets web applications provide a simple interface for users to browse, download, and upload S3-stored data. It integrates with AWS Amplify's JavaScript and React libraries, allowing developers to control access based on user identity and customize the design to fit their app's branding.Juniper jumps into Wi-Fi 7 with enterprise switches, access pointsJuniper has introduced new EX Series switches and Mist Wi-Fi 7 access points for enterprise wireless networks, offering higher speeds, lower latency, and broader range. The EX4400 switches support both Wi-Fi 6E and Wi-Fi 7 and are managed via the AI-powered Mist Cloud, which helps detect and resolve network issues. Juniper’s new AP47 Series access points offer advanced features like dual-5GHz or dual-6GHz operation and AI-based channel management.🛠️HackHub: Best Tools for Cloudkardinal: lightest-weight way to spin up dev and test environments in KubernetesCreate lightweight, temporary development environments within a shared Kubernetes cluster, making testing and development more efficient. It allows developers to spin up tailored, on-demand "flows"—ephemeral environments that use minimal resources by deploying only the necessary services for feature development.kubeblocks: control plane software that runs and manages databases, message queues on K8s.KubeBlocks is an open-source control plane software designed to simplify the management of multiple database engines on Kubernetes (K8s). It uses a unified set of APIs to manage various types of databases, such as MySQL, PostgreSQL, Redis, and Kafka, reducing the need to learn individual database operators.flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solutionFlipt is a cloud-native, GitOps-enabled feature management solution designed to help organizations separate feature releases from deployments, allowing for safer, more controlled updates. It can be integrated into existing infrastructure to avoid third-party latency and is built with high-performance DevOps teams in mind.kubecolor: Colorize your kubectl outputKubecolor is a simple wrapper for the kubectl command-line tool that adds color to its output, making it easier to read and interpret. It enhances the standard kubectl by colorizing logs, tables, and other outputs without changing the actual content. Kubecolor supports custom themes, including options for light backgrounds and colorblind-friendly themes.aws-mine: AWS honey token manageraws-mine is a project designed to create "honey tokens" for AWS, which are fake AWS access keys placed in various locations to lure potential attackers. If someone uses these keys, the system sends a notification within about four minutes, allowing you to investigate the possible compromise. Built with AWS Amplify for easy deployment, users can manage their access through Amazon Cognito and receive alerts via Amazon SNS when the keys are accessed.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

European grocery store becomes cloud services providerCloudPro #65: IBM acquires Kubecost[Sponsored] Use AI to 10X your productivity & efficiency at work with AI (free bonus)Save your free spot here (seats are filling fast!) ⏰⭐MasterclassA Guide to Kubernetes Network PoliciesDockerfile Instructions - ADD vs. COPYHow to add new worker node to existing Kubernetes clusterHow I Reduced Docker Image Size from 588 MB to Only 47.7 MBAmbient mesh: Can sidecar-less Istio make your application faster?🔍Secret KnowledgeOops, I Deleted the AWS Auth RolesRising Incidents on Git PlatformsHow Postgres stores data on diskHow We Integrate a New Service in Under 1 Hour for 25 ClustersEleventeen ways to delete an AWS resource⚡TechwaveEuropean grocery store becomes cloud services providerIBM acquires KubecostIntroducing Pulumi Insights 2.0Linus Torvalds advises open-source developers to pursue meaningful projects, not hypeJFrog Extends GitHub Alliance to Provide Unified Dashboard🛠️HackhubApeman: AWS attack path management toolCyphernetes: A Kubernetes Query LanguageDesed:A command-line tool for complex sed scriptsKueue: Kubernetes-native Job QueueingAWS CloudFormation Starterkit💡Recommended Reading: Implementing GitOps with KubernetesCheers,Shreyans SinghEditor-in-ChiefJoin Roman Lavrik from Deloitte Snyk hosted DevSecCon 2024Snyk is thrilled to announce DevSecCon 2024, Developing AI Trust Oct 8-9, a FREE virtual summit designed for DevOps, developer and security pros of all levels.Join Roman Lavrik from Deloitte, among many others, and learn some presciptive DevSecOps methods for AI-powered development.Save your spot⭐MasterClass: Tutorials & GuidesA Guide to Kubernetes Network PoliciesIn Kubernetes, network policies control the traffic between pods, ensuring secure communication within the cluster. There are two main types: Layer 4 (L4) and Layer 7 (L7) policies. L4 policies manage traffic at the transport layer (e.g., TCP/UDP) based on IP addresses and ports, while L7 policies operate at the application layer (e.g., HTTP) with more fine-grained control over communication between services. L7 policies often require a service mesh like Linkerd, which adds features like mutual TLS (mTLS) for encrypted communication.Dockerfile Instructions - ADD vs. COPY`COPY` is simple and secure, only transferring files from the local build context to the image. In contrast, `ADD` offers extra functionality, such as downloading files from URLs or automatically extracting compressed archives. However, this added flexibility introduces complexity and potential security risks. Best practice recommends using `COPY` for most cases due to its straightforwardness, reserving `ADD` for situations where its unique features are necessary.How to add new worker node to existing Kubernetes clusterTo add a new worker node to an existing Kubernetes cluster, start by setting up a new Ubuntu 24.04 instance and configuring its hostname and `/etc/hosts` file. Disable swap memory, load necessary kernel modules, and install containerd as the container runtime. Add the Kubernetes APT repository, then install Kubernetes components like kubeadm, kubelet, and kubectl. On the control plane node, generate a kubeadm join command with a token. Run this command on the new worker node to join the cluster. Finally, verify the addition by checking the nodes from the control plane using `kubectl get nodes`.How I Reduced Docker Image Size from 588 MB to Only 47.7 MBTo significantly reduce a Docker image size, using multi-stage builds is key. In this case, a Flask app's image size was reduced from 588 MB to just 47.7 MB by switching to the lightweight Python 3.9-alpine image and using a multi-stage build approach. Multi-stage builds allow you to separate the build and runtime environments, keeping only essential runtime dependencies in the final image. Additionally, minimizing the number of layers by combining commands, using a `.dockerignore` file to exclude unnecessary files, and optimizing the Dockerfile structure contributed to this impressive 91.89% reduction.Ambient mesh: Can sidecar-less Istio make your application faster?Ambient mode in Istio, introduced in 2022, allows a sidecar-less architecture that can sometimes make applications faster. In traditional service meshes, adding latency is expected, but tests with ambient mode showed slightly improved performance in some cases, like the Bookinfo application's details service. This is partly because of more efficient connection handling and reduced syscalls in ambient mode, which offsets the overhead of extra hops via lightweight ztunnels.🔍Secret Knowledge: Learning ResourcesOops, I Deleted the AWS Auth RolesThe author, while managing an EKS (Elastic Kubernetes Service) cluster using Terraform, accidentally deleted the AWS authentication roles, which are crucial for accessing the cluster. This resulted in losing access to the EKS cluster. The fix involved manually restoring access by modifying the EKS API access configuration via the AWS Console, re-adding the necessary admin roles, and regenerating the `aws-auth` config map.Rising Incidents on Git PlatformsIn 2023, incidents affecting popular DevOps platforms like GitHub, Bitbucket, GitLab, and Jira increased, with issues such as RepoJacking, security vulnerabilities, and performance disruptions. GitHub saw a rise in attacks, with hackers exploiting vulnerabilities and hosting malware. Atlassian products like Bitbucket and Jira faced security flaws, with Jira experiencing a significant increase in incidents. GitLab suffered from performance issues and security breaches, including a major Proxyjacking attack.How Postgres stores data on diskPostgres stores data on disk in a well-organized, file-based structure within a directory, typically located at `/var/lib/postgresql/data`. Inside this directory, you'll find folders like `base/`, where actual database data for each database is stored, and `pg_wal/`, which holds the Write-Ahead Log (WAL) files that help recover data after crashes. Each table and database object is ultimately represented by files in these directories. PostgreSQL uses clever abstractions to manage data, such as snapshots for transactions, dynamic shared memory for handling multiple processes, and special mechanisms like tablespaces for physically separating certain data.How We Integrate a New Service in Under 1 Hour for 25 ClustersThe article describes how a team integrated a new service called Otterize across 25 clusters in under an hour, emphasizing that while the technical setup was quick, the lengthy licensing process took over four months. The integration involved automating several steps using GitOps and tools like Argo CD to avoid manual errors. Key tasks included creating an organization and environment, inviting users, integrating with Kubernetes, securely managing credentials, and deploying the setup through a script.Eleventeen ways to delete an AWS resourceOur goal is to reduce AWS costs, but the deletion methods vary widely, often leaving users frustrated. They categorize deletion patterns, from simple one-click deletes to more complex confirmations that require typing specific phrases or acknowledging consequences. Ultimately, AWS should standardize its deletion processes to improve user experience and security, and they call for more data on user behavior during these actions.⚡TechWave: Cloud News & AnalysisEuropean grocery store becomes cloud services providerLidl, through its parent company Schwarz Group, unintentionally entered the competitive world of cloud computing when it built its own cloud system in 2021 to meet internal needs. As other German businesses sought alternatives to U.S. and Chinese cloud providers, Schwarz Group recognized a demand for data services with a focus on European data privacy standards. This led to the creation of Schwarz Digits, which now provides cloud and cybersecurity services, attracting major clients like SAP and Bayern Munich. While competing with giants like Amazon and Google, Schwarz Digits differentiates itself with a focus on digital sovereignty and data protection.IBM acquires KubecostIBM has acquired Kubecost, a startup that helps companies optimize and monitor their Kubernetes clusters for cost efficiency. Kubecost, known for its widely adopted Kubernetes cost management tool and its open-source project OpenCost, will enhance IBM’s FinOps capabilities. Kubecost will likely be integrated into IBM's FinOps Suite and potentially its OpenShift platform.Introducing Pulumi Insights 2.0Pulumi Insights 2.0 expands beyond just Pulumi-managed infrastructure to provide visibility into all cloud resources, offering powerful tools for assessing security, efficiency, and management. It introduces new features like comprehensive infrastructure scanning, visual explorers, and dashboards to help organizations manage their cloud environments more effectively. Insights 2.0 integrates with Pulumi’s Infrastructure-as-Code (IaC) tools, making it easier to bring unmanaged infrastructure under IaC.Linus Torvalds advises open-source developers to pursue meaningful projects, not hypeAt the Open Source Summit Europe, Linus Torvalds encouraged open-source developers to focus on meaningful projects rather than chasing trends and hype. While discussing the latest Linux kernel updates, he emphasized that progress in Linux remains steady, even if not always exciting, with a focus on reliability. Torvalds also praised the ongoing evolution of Linux and the wider open-source ecosystem, noting its democratizing effect for new developers.JFrog Extends GitHub Alliance to Provide Unified DashboardJFrog and GitHub have expanded their partnership to provide developers with a unified platform for better security and productivity. This integration offers a consolidated view of project statuses and security through tools like GitHub's Copilot chat and JFrog’s Advanced Security features. Developers can now get insights on third-party packages, track vulnerabilities earlier, and navigate between code and the binaries it produces seamlessly.🛠️HackHub: Best Tools for CloudApeman: AWS attack path management toolProject Apeman is an AWS attack path management tool that helps analyze and manage AWS security data. To set it up, you need Docker, Python, and a virtual environment. Once the system is initialized, Apeman gathers AWS account data, including authorization details and ARNs, which are then ingested into a graph database for analysis.Cyphernetes: A Kubernetes Query LanguageCyphernetes is a Cypher-inspired query language for Kubernetes, simplifying complex Kubernetes operations with intuitive, SQL-like queries. It allows developers to easily manage Kubernetes resources by expressing relationships between them, such as connecting deployments to services and ingresses.Desed:A command-line tool for complex sed scriptsDesed is a command-line tool designed to help debug and understand complex `sed` scripts. It allows users to step through their scripts, both forwards and backwards, preview how substitute commands will affect the pattern space, and set breakpoints to examine the program's state. Desed also supports hot reloading, so changes to the source code can be instantly applied without restarting the debugger.Kueue: Kubernetes-native Job QueueingKueue is a Kubernetes-native job queueing system that manages when jobs start and stop based on a variety of factors, such as priorities and resource availability. It offers features like job management with FIFO strategies, resource fair sharing, dynamic resource reclaim, and integration with popular job types like BatchJob and Kubeflow training jobs.AWS CloudFormation StarterkitAn AWS CloudFormation starterkit including CI/CD and dev tools that allow you to securely and quickly deploy CloudFormation stacks on your AWS account.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Elasticsearch is Open Source, AgainCloudPro #64: Introducing OpenAI o1Hack the Cybersecurity InterviewPrepare for cybersecurity job interviews across various roles, from entry-level to expert positions.It covers topics like answering technical and behavioral questions, understanding different cybersecurity roles and developing important soft skills.It includes information on salaries, career paths, and how to find jobs in tough markets.Get It for $35.99 $24.99⭐MasterclassDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSKubernetes and Access Management API, the new authentication in EKSHow Netflix solved the issue with Java 21 virtual threadsDoes Talos Kubernetes and Omni live up to the hype?Understand your Kubernetes cost drivers and the best ways to rein in spending🔍Secret KnowledgeHack the Cybersecurity InterviewTroubleshooting: Terminal LagMonitor these Kubernetes signals to help rightsize your fleetGetting Started with Cilium Service Mesh on Amazon EKSHow AppsFlyer migrated from Kafka to Kubernetes using Karpenter⚡TechwaveIntroducing OpenAI o1Elasticsearch is Open Source, AgainOracle to offer 131,072 Nvidia Blackwell GPUs via its cloudWhy eBPF is critical and how it’s getting betterJuniper adds AI cloud services to its Apstra data center software🛠️HackhubHigh-performance server for NATS.ioA collection of Bash One-Liners and terminal tricksdistributed key value NoSQL database that uses RocksDB as storage engineBuild, Share and Run Both Your Kubernetes Cluster and Distributed ApplicationsRun your deep learning workloads on Kubernetes more easily and efficientlyCheers,Shreyans SinghEditor-in-ChiefForward to a Friend⭐MasterClass: Tutorials & GuidesDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSThis article explains how to deploy and manage a PostgreSQL database on Kubernetes using Amazon EKS. It combines CloudNativePG, a PostgreSQL operator, with Ceph Rook, a storage orchestrator, to ensure data persistence and high availability in a Kubernetes environment. A step-by-step guide is provided for deploying and configuring these tools, including using Helm charts, setting up storage with Ceph, and monitoring the database cluster.Kubernetes and Access Management API, the new authentication in EKSIn AWS EKS (Elastic Kubernetes Service), the new authentication and access management system simplifies how users and worker nodes access Kubernetes clusters. Previously, access was managed using an "aws-auth" ConfigMap, which could be complex and prone to errors. Now, AWS introduces the EKS Access Management API, allowing users to authenticate through AWS IAM directly and authorize Kubernetes actions via Kubernetes RBAC. This approach removes the need for managing ConfigMaps manually, offers predefined EKS Access Policies, and enhances security by eliminating hidden root users. Additionally, integration with Terraform makes access control easier to implement and manage.How Netflix solved the issue with Java 21 virtual threadsIn Java 21, Netflix encountered an issue with virtual threads, which are lightweight threads designed to improve concurrency by suspending and resuming automatically. The problem involved some Netflix services using Java 21, Spring Boot 3, and Tomcat, where servers stopped processing requests due to sockets stuck in a `CLOSE_WAIT` state. Virtual threads were getting "pinned" to operating system threads while waiting to acquire locks. Since all OS threads became blocked, Tomcat couldn't process incoming requests, causing the system to hang. The underlying issue was traced to locking mechanisms in virtual threads, leading to thread contention and blocked OS threads.Does Talos Kubernetes and Omni live up to the hype?Talos Kubernetes and Sidero Omni live up to the hype by providing an intuitive and efficient way to set up and manage Kubernetes clusters. With Omni, you can easily create a Talos cluster without needing to access your virtual machines directly, making the process more streamlined. Setting up clusters, scaling nodes, and even upgrading Kubernetes versions are straightforward, with minimal manual intervention required. While there are some areas for improvement, like simplifying static IP configuration, the overall experience is highly positive.Understand your Kubernetes cost drivers and the best ways to rein in spendingTo reduce Kubernetes-related costs, it's important to monitor key cost drivers such as CPU, memory, storage, and networking. Costs are driven by resource usage and the rate at which they are consumed, so reducing unnecessary usage and optimizing resource allocation is key. Over-provisioning, idle resources, and inefficient scaling are common cost culprits. Regularly adjusting resource requests, leveraging auto-scaling tools like Horizontal Pod Autoscaler, and monitoring metrics with tools like Grafana and Prometheus can help optimize usage.🔍Secret Knowledge: Learning ResourcesHack the Cybersecurity Interview"Hack the Cybersecurity Interview, Second Edition" is a comprehensive guide designed to help individuals prepare for interviews across a wide range of cybersecurity roles. The book covers technical and behavioral interview questions for positions like cybersecurity engineer, penetration tester, and CISO, while also offering tips on personal branding, stress management, and negotiation. It provides real-world advice and industry insights, making it an essential resource for anyone looking to succeed in the competitive field of cybersecurity.Troubleshooting: Terminal LagIn this troubleshooting session, Tavis Ormandy investigates why launching the xterm terminal on his Windows machine is significantly slower compared to Fedora. He identifies that Windows applies an animation effect that delays interaction with the terminal. Through a series of tests and debugging, he discovers that the X server software (X410) adds unnecessary animation effects, which can’t be disabled directly. He uses a debugger to bypass the issue, improving the performance slightly. After further optimizations with features and caching processes, he brings the Windows terminal's performance closer to Fedora’s, significantly reducing the lag.Monitor these Kubernetes signals to help rightsize your fleetTo ensure your Kubernetes environment is both cost-efficient and sustainable, it's crucial to monitor signals like CPU, memory, disk I/O, and network utilization. Over-provisioning leads to wasted resources and high costs, while under-provisioning can degrade performance. Watch for indicators such as high resource usage, slow application performance, or low utilization to fine-tune your setup. Tools like Prometheus and Grafana, along with autoscaling, can help you dynamically adjust resources, ensuring optimal balance, reduced costs, and improved sustainability.Getting Started with Cilium Service Mesh on Amazon EKSThe blog post explains how Cilium, an open-source networking and security solution powered by eBPF, enhances network connectivity between workloads in Amazon EKS (Elastic Kubernetes Service). Cilium provides advanced networking, load balancing, encryption, and observability without the need for sidecar proxies. It integrates seamlessly with Amazon EKS to improve microservice communication, multi-cluster networking, and network policy enforcement. Cilium Service Mesh, built into Cilium, leverages eBPF and Envoy to offer high performance and low overhead for traffic management, security, and monitoring.How AppsFlyer migrated from Kafka to Kubernetes using KarpenterAppsFlyer, a global leader in mobile attribution, migrated their Kafka infrastructure to Kubernetes using Amazon EKS, simplifying management and improving performance. By switching from EC2 instances to Graviton-powered nodes, they achieved a 75% increase in throughput, 58% better write I/O, and reduced costs by 30%. AWS solutions like Strimzi Kafka Operator, Rancher’s Local Path Provisioner, and Karpenter autoscaler helped optimize local storage management and scaling. This transition cut CPU core usage in half and enhanced AppsFlyer's Kafka cluster’s scalability, efficiency, and resilience.⚡TechWave: Cloud News & AnalysisIntroducing OpenAI o1OpenAI has introduced the "OpenAI o1" series, a new set of AI models designed to focus more on reasoning through complex problems, such as those in science, coding, and math. These models think more carefully before responding and perform significantly better than previous models in areas like math, coding competitions, and complex scientific tasks. Alongside the main "o1-preview" model, there is also a smaller, cheaper "o1-mini" model aimed at developers.Elasticsearch is Open Source, AgainElasticsearch is officially open source again as Elastic has added the AGPL license alongside its existing licenses (ELv2 and SSPL). This move allows Elasticsearch to be called open source under an OSI-approved license, clearing up any confusion caused when Elastic changed its licensing three years ago due to conflicts with AWS. While the license change led to a fork by Amazon, Elastic's partnership with AWS has strengthened, and now users have more licensing options without any impact on current usage.Oracle to offer 131,072 Nvidia Blackwell GPUs via its cloudOracle has announced it will offer 131,072 Nvidia Blackwell GPUs via its Oracle Cloud Infrastructure (OCI) Supercluster, starting in 2025, to support large language model (LLM) training and other AI use cases. This offering aims to meet the growing demand for GPUs, which are essential for generative AI development but in short supply due to limited availability of high-bandwidth memory (HBM). .Why eBPF is critical and how it’s getting bettereBPF (extended Berkeley Packet Filter) is a crucial open-source technology for Linux, providing powerful capabilities for networking, monitoring, and security by allowing safe execution of code in the kernel. It enhances network visibility, reduces patching cycles, and improves performance monitoring. Netflix, for example, uses eBPF for efficient traffic management and security.Juniper adds AI cloud services to its Apstra data center softwareJuniper Networks has updated its Apstra data center software with new AI-powered features, including a cloud-based suite called Apstra Cloud Services and the new 5.0 version of the software. Apstra uses AI to manage network configurations, ensure security policies, and monitor performance across both physical and virtual infrastructures. It now includes App/Service Awareness and Impact Analysis to help data center operators monitor application performance and quickly address issues.🛠️HackHub: Best Tools for Cloudnats-io/nats-serverNATS is a high-performance, cloud-native messaging system designed for modern distributed systems, offering secure and efficient communication between digital services, devices, and systems. It supports over 40 client languages.onceupon/Bash-Oneliner"Bash-Oneliner" is a blog focusing on simple and effective bash commands for tasks like data parsing and Linux system maintenance. The commands are compatible with systems like Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac, and CentOS. The blog covers topics such as variable manipulation, system management, math operations, and networking.apache/kvrocksApache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with the Redis protocol. It aims to reduce memory costs and increase capacity compared to Redis. Kvrocks supports key features like asynchronous replication, high availability with Redis Sentinel, and a centralized cluster management system that works with any Redis cluster client.sealerio/sealerSealer is a tool that simplifies the delivery of distributed applications by packaging a Kubernetes cluster and all application dependencies into a "ClusterImage." A ClusterImage functions similarly to a Docker image, containing everything needed to run the application, such as container images, YAML files, and Helm charts. Users can write a "Kubefile" to build this image and a "Clusterfile" to describe how to run it.kubedl-io/kubedlKubeDL is a CNCF sandbox project that simplifies running deep learning workloads on Kubernetes. It offers features like unified scheduling for training and inference, advanced optimization, and native model tracking using Kubernetes Custom Resource Definitions (CRDs).📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

CloudPro Special EditionCloudPro Special: Mastering Serverless Architecture- A Concise GuideAfter the last CloudPro Special, a lot of subscribers messaged me that I should do it more often. So here we are, with the next CloudPro Special. Today, I’ll talk about Serverless Architecture.There are eight sections:1. Introduction to Serverless Architecture2. Designing Serverless Solutions3. Using Serverless with AWS4. Serverless in the Bigger Picture5. Real-world Use Cases6. Monitoring Serverless Apps7. Pros and Cons8. Conclusion`Each section has additional learning resources:Cloud Computing Demystified for Aspiring ProfessionalsArchitecting Cloud-Native Serverless SolutionsMulti-Cloud Strategy for Cloud ArchitectsAWS CDK in PracticeMulti-Cloud Handbook for DevelopersAWS for Solutions ArchitectsMastering Amazon EC2Solutions Architect's HandbookAWS Certified Developer Associate Certification and BeyondAWS Observability HandbookAny feedback or questions, just reply back to this email and let me know. Without further delay, let's jump into today's CloudPro Special!Cheers,Shreyans SinghEditor in ChiefIntroduction to Serverless ArchitectureServerless architecture is a new way to build and run apps without worrying about servers. Despite its name, there are still servers involved, but cloud providers like Amazon or Google manage them for you. This means you can focus on writing code instead of managing hardware.Serverless is the latest step in cloud computing. It started with physical machines, then moved to services where you rent virtual servers, and now we have serverless where you just run your code without thinking about the servers at all.💡Learning ResourceTo learn more about serverless and cloud computing, check out "Cloud Computing Demystified for Aspiring Professionals" ($24.99 $35.99). It explains these concepts in simple terms.Learn more about cloud computing and serverlessDesigning Serverless SolutionsWhen building serverless apps, keep these things in mind:📌Make your functions independent (they shouldn't rely on saved information)📌Build your app around events and triggers📌Break your app into small, separate services📌Be aware of "cold starts" which can slow things down📌Make sure your functions finish quickly (there are time limits)Remember, serverless isn't always the cheapest option. It works best for apps with unpredictable usage. For apps with steady, predictable usage, traditional servers might be cheaper.💡Learning ResourceTo learn more about designing serverless apps, I recommend "Architecting Cloud-Native Serverless Solutions" ($24.99 $35.99). It's a helpful guide for building serverless apps on different cloud platforms.Learn more about designing serverless solutionsIf you want to use serverless with multiple cloud providers, check out "Multi-Cloud Strategy for Cloud Architects" ($29.99 $43.99). It helps you understand how to use serverless across different providers.Learn about using multiple cloud providersUsing Serverless with AWSAWS offers many serverless tools. Here are the main ones:📌AWS Lambda: Run your code📌Amazon API Gateway: Create and manage APIs📌AWS Step Functions: Coordinate multiple functions📌Amazon EventBridge: Build apps that respond to events📌AWS SAM: Make serverless development easierOne useful tool for deploying serverless apps on AWS is the AWS Cloud Development Kit (CDK). It lets you set up your cloud infrastructure using regular programming languages.💡Learning ResourceTo learn how to use AWS CDK, I suggest "AWS CDK in Practice" ($27.98 $39.99). It teaches you how to build complex serverless apps easily.Learn about AWS CDKTo compare serverless options across different providers, check out "Multi-Cloud Handbook for Developers" ($27.98 $39.99).Learn about different cloud providersServerless in the Bigger PictureWhile serverless is great, it's not perfect for every situation. Sometimes, traditional cloud services might work better. For example, if you have long-running processes or steady workloads, using regular servers or containers might be better.To understand when to use serverless and when to use other options, we recommend these resources:💡Learning ResourceLearn about AWS architectureLearn about traditional cloud computingLearn about different cloud architecturesReal-world Use CasesMany companies use serverless successfully. Here are a few examples:📌Coca-Cola: Used serverless for vending machines, cutting costs by 65%.📌Netflix: Uses serverless for tasks like processing videos, handling billions of events daily.📌Zalora: Switched to serverless, reducing infrastructure costs by 60%.These companies learned some important lessons:📌Start small and gradually move more parts of your app to serverless📌Use good monitoring tools📌Use code to manage your infrastructure📌Keep improving your functions' performance and cost💡Learning ResourceTo learn more about real-world serverless uses, check out "AWS Certified Developer Associate Certification and Beyond" ($27.98$39.99). It gives practical insights into building serverless solutions on AWS.Learn about serverless development on AWSMonitoring Serverless AppsKeeping an eye on serverless apps is different from traditional apps. You need to watch:📌How different functions work together📌How long it takes for functions to start and run📌How much your functions cost to run📌Logs from all your functions💡Learning ResourceTo learn how to monitor serverless apps on AWS, I recommend the "AWS Observability Handbook" ($27.98 $39.99) It shows you how to use AWS tools to watch, track, and get alerts about your serverless apps.Learn about AWS monitoringPros:📌Can be cheaper for some types of apps📌Automatically handles more users📌Lets you release new features faster📌Lets developers focus on code📌No need to manage serversCons:📌Can be slow to start sometimes📌Limited run time (usually max 15 minutes)📌Can be harder to find and fix problems📌Might be hard to switch to a different provider📌Less control over the underlying systemConclusionServerless is a powerful way to build apps that can handle any number of users and potentially save you money. It lets developers focus on writing code instead of managing servers.Remember, serverless isn't always the best choice. Always consider your specific needs when deciding whether to use serverless.To learn more, check out the learning resources mentioned in this guide. They'll help you become an expert in serverless computing.Happy learning, and good luck with your serverless projects!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Red Hat Enterprise Linux AI Now Generally AvailableCloudPro #63: Google Cloud has launched Memorystore for Valkey200+ hours of research on AI-led career growth strategies & hacks packed in 3 hoursThe only AI Crash Course you need to master 20+ AI tools, multiple hacks & prompting techniques in just 3 hoursYou’ll save 16 hours every week & find remote jobs using AI that will pay you upto $10,000/moGet It Here⭐Masterclass[Sponsored] 200+ hours of research on AI-led career growth strategies & hacks packed in 3 hoursThe Kubernetes gap in CNAPPUnlock Kubernetes Savings with Kubecost’s Automated ActionsHow WebAssembly components extend the frontiers of Kubernetes to multi-cloud, edge, and beyondHow to migrate an observability platform to open-source and cut costs🔍Secret KnowledgeImplementing GitOps with Kubernetes: Automate, manage, scale, and secure infrastructure and cloud-native applications on AWS and AzureComplete Guide to Logging in Golang with slogScaling Prometheus with ThanosAutomated container CVE and vulnerability patching using Trivy and CopaceticSelf-signed Root CA in Kubernetes with k3s, cert-manager and traefik⚡TechwaveRed Hat Enterprise Linux AI Now Generally AvailableKubernetes 1.31: Streaming Transitions from SPDY to WebSocketsGoogle Cloud has launched Memorystore for ValkeyPalo Alto Networks acquires IBM QRadar SaaS assetsBroadcom Adds On-Premises Edition of Project Management Application🛠️HackhubProduction-ready Kubernetes distribution for both public and private cloudApplication Performance Monitoring SystemGraceful shutdown and Kubernetes readiness / liveness checks for any Node.js HTTP applicationsToolkit for Integrating with your kubernetes dev environment more efficientlyBackup your Kubernetes Stateful ApplicationsCheers,Shreyans SinghEditor-in-ChiefLive Webinar: The Power of Data Storytelling in Driving Business Decisions (September 10, 2024 at 9 AM CST)Data doesn’t have to be overwhelming. Join our webinar to learn about Data Storytelling and turn complex information into actionable insights for faster decision-making.Click below to check the schedule in your time zone and secure your spot. Can't make it? Register to get the recording instead.REGISTER FOR FREEForward to a Friend⭐MasterClass: Tutorials & GuidesThe Kubernetes gap in CNAPPInitially, CNAPPs focused on integrating various cloud security tools and supporting enterprises during early cloud adoption. As a result, their Kubernetes protection often lacks depth and focuses mainly on surface-level issues like container vulnerabilities, without addressing the complexities of Kubernetes clusters, such as control plane security or runtime policies. This has led to a false sense of security in cloud environments, as CNAPPs fail to offer robust Kubernetes-specific features.Unlock Kubernetes Savings with Kubecost’s Automated ActionsKubecost's new automated actions help users save money in their Kubernetes environments by optimizing resource usage with minimal effort. With features like automated request sizing, cluster turndown, and namespace turndown, Kubecost identifies inefficiencies like over-provisioned containers and shuts down unused clusters or namespaces. Users can set schedules for automating these actions, reducing waste and freeing up resources.How WebAssembly components extend the frontiers of Kubernetes to multi-cloud, edge, and beyondWebAssembly (Wasm) components enable Kubernetes to extend seamlessly across multi-cloud, edge, and other distributed environments by providing a lightweight, portable way to run applications across any architecture. Wasm components, similar to containers, can be written in various languages and connected through shared APIs, allowing for greater flexibility and efficiency. By integrating with Kubernetes through wasmCloud, a Wasm-native orchestrator, organizations can enhance their cloud-native setups without changing existing infrastructure.How to migrate an observability platform to open-source and cut costsMigrating an observability platform to open-source can significantly reduce costs while maintaining control over telemetry data, but it requires careful planning and execution. This process involves identifying essential telemetry data, selecting an open-source stack for logs, metrics, and traces, conducting proofs-of-concept (POCs) across different systems, and ensuring compatibility with various architectures, such as microservices. The migration also includes reconfiguring alerts and dashboards, validating the new setup, and updating related systems like notification and incident management tools.🔍Secret Knowledge: Learning ResourcesImplementing GitOps with Kubernetes: Automate, manage, scale, and secure infrastructure and cloud-native applications on AWS and AzureThis book provides practical guidance on using GitOps to automate and manage Kubernetes deployments in cloud-native environments like AWS and Azure. It explains core GitOps principles, tools like Argo CD and Flux, and strategies for implementing CI/CD pipelines. The book also covers infrastructure automation with Terraform, security best practices, and observability while addressing cultural transformations in IT for GitOps adoption. By the end, readers will have skills to apply GitOps in scaling, monitoring, and securing Kubernetes deployments efficiently.Complete Guide to Logging in Golang with slogIn Golang, structured logging can be efficiently implemented using the `slog` package, introduced in version 1.21. `slog` allows for more organized and detailed log entries by formatting logs as key-value pairs, making them easier to search, filter, and analyze. The package provides flexibility with logging levels (like Debug, Info, Warn, and Error) and supports both text-based and JSON-formatted output. Key components include Loggers, Records, and Handlers, which define how logs are created, stored, and processed.Scaling Prometheus with ThanosScaling Prometheus with Thanos allows for long-term storage, cost savings, and a global view of metrics in large environments. While Prometheus is great for short-term monitoring, it struggles with long-term storage and querying across multiple clusters. Thanos extends Prometheus by using components like Thanos Query, Sidecar, and Store Gateway to enable scalable, highly available storage through object stores, reducing Prometheus's resource consumption. It also supports downsampling to optimize storage and query performance.Automated container CVE and vulnerability patching using Trivy and CopaceticAutomating container vulnerability patching with Trivy and Copacetic (copa) helps protect your applications from potential attacks by scanning and patching container images automatically. Trivy scans container images for vulnerabilities, generating a report in JSON format, while Copacetic reads this report and patches the container image based on detected vulnerabilities. Once patched, the image is rebuilt and rescanned to ensure all vulnerabilities have been fixed.Self-signed Root CA in Kubernetes with k3s, cert-manager and traefikIn Kubernetes with k3s, cert-manager, and Traefik, you can create a self-signed root Certificate Authority (CA) to manage TLS certificates locally, useful when your cluster isn't exposed to the internet (e.g., no Let's Encrypt). The process involves setting up cert-manager to automate the issuance, renewal, and secret management of these certificates. You first create a self-signed root CA, which then signs an intermediate CA, and that intermediate CA signs leaf certificates for your services. This setup allows your services to have trusted certificates locally.Developing for iOS? Setapp's 2024 report on the state of the iOS market in the EU is a must-seeHow do users in the EU find apps? What's the main source of information about new apps? Would users install your app from a third-party app marketplace?Set yourself up for success with these and more valuable marketing insights in Setapp Mobile's report iOS Market Insights for EU.Get Insights free⚡TechWave: Cloud News & AnalysisRed Hat Enterprise Linux AI Now Generally AvailableRed Hat Enterprise Linux (RHEL) AI is now available, providing an open-source platform for developing and running generative AI models across hybrid cloud environments. It combines efficient models, such as the Granite LLM family, and tools like InstructLab to help align models with specific business needs. RHEL AI allows domain experts, not just data scientists, to contribute to AI models, making them more accessible and cost-effective.Kubernetes 1.31: Streaming Transitions from SPDY to WebSocketsIn Kubernetes 1.31, the default streaming protocol used by kubectl has shifted from the outdated SPDY protocol to the more modern and widely supported WebSocket protocol. Streaming protocols in Kubernetes enable persistent, real-time communication between the client and server, which is useful for operations like running commands inside a container. The switch to WebSockets improves compatibility with modern proxies and gateways, ensuring commands like `kubectl exec`, `kubectl cp`, and `kubectl port-forward` function smoothly across different environments.Google Cloud has launched Memorystore for ValkeyGoogle Cloud has launched Memorystore for Valkey, a fully managed, high-performance key-value service that is 100% open-source. Valkey 7.2 is compatible with Redis 7.2 and offers features like zero-downtime scaling, persistence, and integration with Google Cloud. It's designed to meet the demand for open-source data management, providing users with an alternative to Redis for use cases like caching and session management. Valkey is gaining popularity due to its performance and scalability, and Google Cloud plans to expand its capabilities further with Valkey 8.0, which promises even better performance and reliability.Palo Alto Networks acquires IBM QRadar SaaS assetsPalo Alto Networks has acquired IBM's QRadar SaaS assets to enhance their joint AI-powered security solutions, aiming to help organizations strengthen their cybersecurity operations. This partnership will simplify threat detection, improve security automation, and deliver next-generation security operations at scale. IBM will support seamless migrations to Palo Alto's Cortex XSIAM platform.Broadcom Adds On-Premises Edition of Project Management ApplicationAt VMware Explore 2024, Broadcom introduced an on-premises version of its Rally project management application, called Rally Anywhere, to give organizations more control over their data. This version is especially valuable for industries with strict regulations or concerns about ransomware targeting SaaS platforms. Rally Anywhere offers an alternative to Atlassian’s Jira, which is discontinuing its on-premises option, and helps organizations meet data sovereignty requirements.🛠️HackHub: Best Tools for Cloudlabring/sealosSealos is a cloud operating system built on the Kubernetes kernel, designed to simplify managing cloud-native applications. It offers quick deployment of distributed applications and high-availability databases like MySQL, PostgreSQL, and MongoDB.apache/skywalkingApache SkyWalking is an open-source Application Performance Monitoring (APM) system designed for microservices, cloud-native, and container-based architectures. It offers end-to-end distributed tracing, service observability, and diagnostic tools, supporting various programming languages like Java, .NET, PHP, and Python.godaddy/terminusTerminus is a Node.js package that helps manage graceful shutdowns and Kubernetes health checks for HTTP applications. Terminus also provides readiness and liveness checks to inform Kubernetes about the service’s health status.alibaba/kt-connectKT-Connect is a tool that helps developers efficiently connect, redirect, and expose local applications to Kubernetes clusters for easier testing and development.stashed/stashStash by AppsCode is a cloud-native backup and recovery solution for Kubernetes workloads, making it easier to back up and restore data like volumes and databases in dynamic Kubernetes environments. It simplifies the backup process using tools like restic and Kubernetes CSI Driver VolumeSnapshotter.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Announcing Terraform Google Provider 6.0.0 CloudPro #62: Kubernetes 1.31 Fine-grained SupplementalGroups control Quick Start Kubernetes Understand what Kubernetes is and why it's essential Learn the inner workings of Kubernetes architecture Get hands-on with deploying and managing applications Set up Kubernetes and containerize applications GET IT FOR $18.99 $12.99 ⭐Masterclass: Unlock the Full Potential of Kubernetes for Scalable Application Management Kubernetes pod and container restarting Better Kubernetes YAML Editing with (Neo)vim Monitoring kubernetes events with kubectl and Grafana Loki Practical Logging for PHP Applications with OpenTelemetry Using 1Password with External Secrets Operator in a GitOps way 🔍Secret Knowledge: Build your own SQS or Kafka with Postgres Revealing the Inner Structure of AWS Session Tokens An Opinionated Ramp Up Guide to AWS Pentesting Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobs Application Availability Depends on Dependencies ⚡Techwave: Kubernetes 1.31: Fine-grained SupplementalGroups control Announcing Terraform Google Provider 6.0.0 New capabilities in VMware Private AI Foundation with NVIDIA GitLab Announces the General Availability of GitLab Duo Enterprise Grafana 11.2 release: new updates for data sources, visualizations, transformations, and more 🛠️HackHub: Best Tools for the Cloud PostgreSQL cloud native High Availability and more Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates Runs and manages databases, message queues, etc on K8s Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources configure kubernetes objects on multiple clusters using jsonnet Cheers, Shreyans Singh Editor-in-Chief Mobile Banking Apps: Secure SDKs Aren’t Enough (Webinar) Is your mobile banking app truly secure? Join our webinar to learn why relying solely on protected SDKs leaves your app vulnerable. Discover real-world scenarios where emerging vulnerabilities can compromise your app despite using a protected SDK. We'll cover multi-layered protection strategies and practical solutions to guard against reverse engineering, tampering, and malware. Gain actionable insights on using obfuscation, data encryption, and real-time application self-protection (RASP) to safeguard your app. Equip yourself with practical solutions to ensure comprehensive app security and safeguard your business from financial and regulatory risks. REGISTER NOW Forward to a Friend ⭐MasterClass: Tutorials & Guides Kubernetes pod and container restarting In Kubernetes, a Pod is the smallest deployable unit, often containing one or more containers. When a container or pod needs to be restarted due to errors or updates, Kubernetes offers several methods to do so. For example, you can restart a Pod by deleting it, and Kubernetes will automatically recreate it if it’s part of a Deployment. Alternatively, you can restart a specific container within a Pod using commands like `kubectl exec` for more precise control. These features allow Kubernetes to maintain high availability and resilience in a cloud environment. Better Kubernetes YAML Editing with (Neo)vim Editing Kubernetes YAML files can be tricky, but using Neovim, a modern version of Vim, can make it much easier. Neovim is lightweight, highly customizable, and integrates well with your terminal, making it ideal for DevOps and platform engineers. By configuring Neovim specifically for YAML files, you can set up features like auto-indentation, syntax highlighting, folding, and autocompletion, all of which help reduce errors and improve efficiency. Monitoring kubernetes events with kubectl and Grafana Loki In Kubernetes, monitoring events is crucial for understanding the status and issues related to Pods, WorkerNodes, and other components. You can use `kubectl` to view these events directly, or you can enhance your monitoring setup by integrating Kubernetes events with Grafana Loki. By capturing events as logs using a tool like the `k8s-event-logger`, which listens to the Kubernetes API, you can store them in Loki, create metrics with RecordingRules, and visualize them in Grafana. Practical Logging for PHP Applications with OpenTelemetry Practical logging for PHP applications using OpenTelemetry involves instrumenting your PHP code to collect and correlate log data with other observability signals like traces and metrics. This approach is particularly useful in microservices-based architectures, where understanding the interactions between different services is crucial for maintaining system stability. By using OpenTelemetry, developers can standardize how telemetry data is collected and exported, reducing complexity. Using 1Password with External Secrets Operator in a GitOps way To manage secrets securely in a GitOps environment using Kubernetes, you can integrate 1Password with the External Secrets Operator. This setup allows you to automatically fetch and inject secrets stored in 1Password into your Kubernetes cluster. By using tools like ArgoCD, Helm, or FluxCD, you can deploy and manage this integration efficiently. The External Secrets Operator pulls secrets from 1Password via 1Password Connect, a proxy that ensures availability and reduces API requests. PACKT TITLES FOR YOU Buy now at $16.99 $10.99 Buy now at $39.99 $27.98 Buy now at $24.99 $16.99 🔍Secret Knowledge: Learning Resources Build your own SQS or Kafka with Postgres You can build your own version of SQS (Simple Queue Service) or Kafka using PostgreSQL by setting up tables and queries that mimic the functionality of these popular message queues and streams. For SQS, you create a table to store messages, with columns that help manage message visibility, delivery attempts, and order. You can then write queries to insert messages, retrieve them while respecting visibility timeouts, and delete them after processing. For Kafka, you expand this setup by storing messages persistently and keeping track of where each consumer group is in the message stream, allowing multiple consumers to process messages independently and in parallel, similar to Kafka's partitioning system. Revealing the Inner Structure of AWS Session Tokens By reverse engineering these tokens, the research team developed tools to analyze and modify them programmatically. This allowed them to uncover previously unknown details about AWS's cryptography and authentication protocols. Their findings showed that while AWS's security measures are robust, understanding the structure of these tokens can help defenders better protect against potential attacks. Additionally, the research raises questions about the privacy and integrity of these tokens. An Opinionated Ramp Up Guide to AWS Pentesting) Lizzie Moratti's "Opinionated Ramp Up Guide to AWS Pentesting" offers a detailed roadmap for becoming proficient in AWS pentesting, emphasizing practical experience over certifications. The guide is tailored for those with a foundational understanding of networking and security, and it stresses the importance of broad knowledge before delving into deeper cloud-specific skills. The guide also touches on industry pitfalls, such as reliance on automated tools and the challenges of cloud pentesting in a fast-evolving environment. Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobs AWS Batch now supports multi-node parallel (MNP) jobs for Amazon EKS, allowing you to gang schedule pods across multiple nodes for tasks that require extensive computation, like machine learning or weather forecasting. Previously, MNP jobs were only available on Amazon ECS. With this update, you can use AWS Batch on EKS to run distributed processing jobs, such as those with Dask, a Python library for parallel computing. The setup involves defining job configurations that include a main node running the scheduler and worker nodes executing the tasks. This approach ensures efficient communication and scaling across nodes, streamlining complex computations in a managed environment. Application Availability Depends on Dependencies Modern applications depend on various services and components, meaning their reliability is tightly linked to the uptime of these dependencies. For example, if an application like Tekata.io needs to maintain 99.9% uptime, but it relies on several services with only 99.9% uptime each, the combined effect could reduce Tekata.io’s overall availability. To hit the desired uptime, dependencies need to have even higher availability. The formula \( A = U^N \) shows that if your application’s target uptime is 99.9% and it has 7 dependencies, each dependency must have an uptime of 99.99% to meet that target. ⚡TechWave: Cloud News & Analysis Kubernetes 1.31: Fine-grained SupplementalGroups control In Kubernetes 1.31, a new feature called `supplementalGroupsPolicy` was introduced to give better control over how supplementary group IDs are handled in Pods. Previously, Kubernetes automatically included group memberships defined in the container’s `/etc/group` file, which could lead to unexpected group IDs being applied and potentially cause security or access issues. With this update, you can now specify a `Strict` policy that only includes the group IDs explicitly set in the Pod's manifest, excluding any additional groups defined in the container image. Announcing Terraform Google Provider 6.0.0 The Terraform Google Provider 6.0.0 introduces several enhancements for better management of Google Cloud resources. Key updates include the option to opt-out of a default label ("goog-terraform-provisioned") that identifies Terraform-managed resources, improved protection against accidental resource deletion with new deletion protection fields, and increased flexibility with longer name prefixes for resources. New capabilities in VMware Private AI Foundation with NVIDIA Key updates in VMware Private AI include a Model Store for secure LLM management, a streamlined deployment process, and new NVIDIA capabilities like NIM Agent Blueprints for custom AI workflows. Future updates will include better GPU management, advanced data indexing and retrieval services, and tools for building AI agents. GitLab Announces the General Availability of GitLab Duo Enterprise GitLab has launched GitLab Duo Enterprise, an AI-powered add-on designed to enhance the software development lifecycle for DevSecOps teams. Priced at $39 per user per month, this tool integrates advanced AI features to improve code generation, security vulnerability detection, and team collaboration. It builds on the capabilities of GitLab Duo Pro by adding enterprise-focused tools like vulnerability resolution, root cause analysis, and AI impact dashboards. Grafana 11.2 release: new updates for data sources, visualizations, transformations, and more Notable additions include support for new data sources like Yugabyte and Amazon Managed Service for Prometheus, updates to visualizations such as standardized tooltips and pagination for state timelines, and improvements in transformations like data transposing and enhanced template variable support. The release also includes better alerting features, integration improvements for OAuth and SAML providers, and a migration assistant for easier transition to Grafana Cloud. 🛠️HackHub: Best Tools for Cloud sorintlab/stolon Stolon is a cloud-native tool designed to manage PostgreSQL databases with high availability, making it suitable for deployment in various environments including Kubernetes and traditional infrastructures. It leverages PostgreSQL's streaming replication and integrates with cluster stores like etcd, Consul, or Kubernetes for leader election and data storage. keel-hq/keel Keel is a lightweight tool for automating updates to Kubernetes deployments without needing complex command-line interfaces or APIs. It integrates directly with Kubernetes and Helm, using labels and annotations to manage updates based on semantic versioning policies. apecloud/kubeblocks KubeBlocks is an open-source tool designed to simplify the management of multiple database types on Kubernetes using a unified set of APIs. Instead of dealing with different operators for each database, KubeBlocks provides a single control plane to manage various databases such as PostgreSQL, Redis, and Kafka. It offers a standardized approach to database lifecycle management, day-2 operations, and observability, with support for backup, recovery, and monitoring. caicloud/cyclone Cyclone is a workflow engine built for Kubernetes that manages end-to-end pipelines without requiring extra dependencies. It operates across various Kubernetes environments, including public, private, and hybrid clouds. Cyclone offers features like DAG graph scheduling, flexible parameterization, and integration with external systems. It supports triggers, multi-cluster execution, multi-tenancy, and automatic resource cleanup. splunk/qbec Qbec is a CLI tool designed for managing Kubernetes objects across multiple clusters or namespaces using jsonnet, a data-templating language. It simplifies Kubernetes configuration management by allowing users to define and deploy objects in various environments efficiently. Qbec is similar to tools like kubecfg and ksonnet. 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}.reverse{display:table;width: 100%;

Github Copilot Autofix: Secure code 3x faster CloudPro #61: How Figma Migrated onto K8s in Less Than 12 months ⭐Masterclass: From Docker Compose to Kubernetes Manifests A hard look at GuardDuty shortcomings Streamlining Keycloak in Kubernetes The hater’s guide to Kubernetes A skeptic's first contact with Kubernetes 🔍Secret Knowledge: Enhancing Bitnami Helm Charts Security Cloudflare adopted OpenTelemetry for logging pipeline Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Did you know the CNCF has an actual cookbook? Not metaphorically! Unfashionably secure: why we use isolated VMs ⚡Techwave: How Figma Migrated onto K8s in Less Than 12 months Github Copilot Autofix: Secure code 3x faster New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Announcing mandatory multi-factor authentication for Azure sign-in GitHub scales on demand with Azure Functions 🛠️HackHub: Best Tools for the Cloud Web tool for database management The devs are over here at devzat, chat over SSH! CloudFormation_To_Terraform Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster. Kubernetes network solution Cheers, Shreyans Singh Editor-in-Chief Forward to a Friend ⭐MasterClass: Tutorials & Guides From Docker Compose to Kubernetes Manifests This blog post provides a beginner-friendly guide for developers transitioning from Docker Compose to Kubernetes manifests, using Minikube for local Kubernetes development. It walks through setting up Minikube, deploying a sample application using Docker Compose, converting Docker Compose files into Kubernetes manifests with Kompose, and finally deploying the application on Kubernetes. The guide emphasizes practical steps, like generating and applying Kubernetes manifests, and validating deployments using the Minikube dashboard. A hard look at GuardDuty shortcomings AWS GuardDuty, while a cornerstone in cloud threat detection, isn't without its flaws. It offers good coverage and deep integration with AWS services, but its limitations in service support, detection latency, and cost can leave gaps in your security posture. Adversarial simulations and benchmarks reveal that GuardDuty can miss critical threats, and its detection can be slow, especially for high-impact, low-volume attacks like S3 ransomware. Streamlining Keycloak in Kubernetes In this blog post, the author, a DevOps Engineer at Tikal, shares how they automated the deployment and management of Keycloak, an open-source identity and access management solution, within a Kubernetes environment. By leveraging Kubernetes’ native capabilities, Helm, and Python, they streamlined the complex configuration process, which typically requires extensive manual adjustments. This approach not only ensures consistency and reduces manual efforts but also enables scalable and repeatable deployments. The hater’s guide to Kubernetes Kubernetes often gets a bad rap for being overly complex, especially for startups with small teams. Critics argue it’s over-engineering for tasks that don't need such a heavyweight solution. The key to avoiding its complexity is to use only the necessary features and ignore the rest. While Kubernetes isn’t for everyone, especially for those needing quick, ephemeral workloads, it's a solid choice if you need the robustness it offers and are careful in its application. A skeptic's first contact with Kubernetes The author’s first real exploration of Kubernetes revealed its core concepts like control loops, services, and workload management, which actually simplify and automate many tasks traditionally done manually. Kubernetes uses controllers to ensure that workloads meet desired states, services to manage network traffic efficiently, and storage management to handle data persistence across pods. While the system has some quirks and limitations, its approach to automating and scaling workloads has proven to be a valuable evolution in managing modern infrastructure. Quick Start Kubernetes The course prepares you to leverage Kubernetes for continuous development and deployment. Whether you're scaling applications to meet demand or ensuring seamless updates with minimal downtime, you'll be equipped with the skills necessary for efficient and effective Kubernetes management. This course is your gateway to becoming proficient in one of the most essential tools in the DevOps toolkit. 🔍Secret Knowledge: Learning Resources Related Titles Enhancing Bitnami Helm Charts Security Bitnami enhanced the security of its Helm charts using Kubescape, an open-source Kubernetes security tool that identifies misconfigurations by comparing configurations to industry best practices. By integrating Kubescape into their build pipelines, Bitnami made significant improvements such as eliminating group root dependencies, configuring immutable filesystems, and reducing misconfigured resources. Cloudflare adopted OpenTelemetry for logging pipeline Cloudflare recently transitioned its logging pipeline from syslog-ng to OpenTelemetry Collector to enhance performance, maintainability, and telemetry insights. This move allowed the team to leverage Go, a language more familiar to their engineers, and integrate better observability through Prometheus metrics. Despite challenges like minimizing downtime during the switch and ensuring compatibility with existing infrastructure, the migration has opened up opportunities for further improvements, such as better log sampling and migration to the OpenTelemetry Protocol (OTLP). Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Josh Grose (ex-Principal PM, Splunk), after three years away from the observability space, was surprised to find that despite companies spending around 30% of their cloud budgets on monitoring, reliability hasn't improved significantly. He observed that even when Service Level Agreements (SLAs) are met, it often comes at the cost of developer productivity and experience. Engineering leaders are frustrated with the high costs and limited improvements in key metrics like Mean Time to Recovery (MTTR) and development speed, leading to the perception that observability has become an expensive and ineffective necessity. Did you know the CNCF has an actual cookbook? Not metaphorically! The "Cloud Native Community Cookbook" is a unique collection of recipes put together by the CNCF and Equinix Metal, born out of the increased time people spent at home during the COVID-19 pandemic. Instead of focusing on cloud technologies, this cookbook brings together food recipes shared by members of the Cloud Native community, originally exchanged in Equinix Metal's Slack channel. Unfashionably secure: why we use isolated VMs While modern cloud architectures often favor shared, multi-tenant environments for efficiency and scalability, Thinkst Canary opts for a less trendy but highly secure approach by using isolated virtual machines (VMs) for each customer. This choice prioritizes security by ensuring that each customer's data and services are completely separated, reducing the risk of cross-customer data breaches. Although this method comes with higher operational costs and complexity, it provides a stronger security boundary, making it easier to manage risks and sleep better at night. ⚡TechWave: Cloud News & Analysis How Figma Migrated onto K8s in Less Than 12 months Figma completed its migration to Kubernetes in under a year by meticulously planning and executing a well-scoped transition. Initially running services on AWS's ECS, Figma faced limitations such as complex stateful workloads and limited auto-scaling. The decision to move to Kubernetes (EKS) was driven by its broader functionality, including support for StatefulSets, Helm charts, and advanced scaling options from the CNCF ecosystem. By Q1 2024, Figma had migrated most core services with minimal impact on users, resulting in enhanced reliability, reduced costs, and a more flexible compute platform. Github Copilot Autofix: Secure code 3x faster Copilot Autofix, now available in GitHub Advanced Security, is an AI-powered tool designed to help developers fix code vulnerabilities more than three times faster than manual methods. It analyzes vulnerabilities, explains their significance, and offers code suggestions for quick remediation. This accelerates the fixing process for both new vulnerabilities and existing security debt, significantly reducing the time and effort required for secure coding. Copilot Autofix is included by default for GHAS customers and also available for open source projects starting in September. New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Kubernetes v1.31 introduces a new alpha feature called "distribute-cpus-across-cores" for the CPUManager's static policy. This option aims to enhance performance by spreading CPUs more evenly across physical cores, rather than clustering them on fewer cores. This reduces contention and resource sharing between CPUs on the same core, which can boost performance for CPU-intensive applications. To use this feature, users need to adjust their Kubernetes configuration to enable it. Currently, it cannot be combined with other CPUManager options, but future updates will address this limitation. Announcing mandatory multi-factor authentication for Azure sign-in Microsoft is making multi-factor authentication (MFA) mandatory for all Azure sign-ins to enhance security and protect against cyberattacks. Starting in the latter half of 2024, Azure users will need to use MFA to access the Azure portal and admin centers, with broader enforcement for other Azure tools like CLI and PowerShell set for early 2025. MFA, which adds an extra layer of security by requiring more than just a password, is shown to block over 99% of account compromises. GitHub scales on demand with Azure Functions GitHub faced scalability issues with its internal data pipeline, which struggled to handle the massive amount of data it collects daily. To address this, GitHub partnered with Microsoft to use Azure Functions' new Flex Consumption plan, which allows serverless functions to scale dynamically based on demand. This solution has enabled GitHub to efficiently process up to 1.6 million events per second, addressing their growth challenges and improving performance with minimal overhead. 🛠️HackHub: Best Tools for Cloud commandprompt/pgmanage PgManage is a modern graphical database client for PostgreSQL, focusing on management features and built on the now-dormant OmniDB project. quackduck/devzat Devzat is a chat service accessible via SSH that replaces the traditional shell prompt with a chat interface, allowing you to connect from any device with SSH capabilities. aperswal/CloudFormation_To_Terraform The CloudFormation to Terraform Converter is a tool that automates the migration of AWS CloudFormation templates to Terraform configuration files. bloomberg/goldpinger Goldpinger monitors Kubernetes networking by making calls between its instances and providing Prometheus metrics for visualization and alerts. ZTE/Knitter Knitter is a Kubernetes CNI plugin that supports multiple network interfaces for pods, allowing custom network configurations across various cloud environments. Buy now at $16.99 $10.99 Buy now at $39.99 $27.98 Buy now at $24.99 $16.99 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}