New Farming Methods in the Epistemological Wasteland of Application Security

New Farming Methods
in the Epistemological
Wasteland of
Application Security
- @wickett

@wickett #ruggeddevops
bit.ly/rdo-farming
Slides

James Wickett
SR. ENGINEER, SIGNAL SCIENCES
AUSTIN, TX
HANDS-ON GAUNTLT BOOK
DEVOPS DAYS GLOBAL ORGANIZER
LASCON ORGANIZER

Application Security Telemetry and Monitoring
Plus Defense!
Application Security for the rest of us
An approach that integrates with devops
organizations doesn't inhibit going fast

Software development is a constant experiment in knowing
Application Security abdicated runtime responsibility and
development responsibility through incoherent
philosophical approaches and fostering silo-thinking
Security now is where Ops was 7 years ago.
Ops found a path to change through devops, security can
too
There are three ways we can add value: at development, at
deploy, at runtime
Summary

Bad-Behavior Driven Development
Weaponizing your CD Pipeline
Application Security Telemetry and Monitoring
Continuous Hardening and Audit
Have a S-BOM! (Software Bill of Materials)
Practices

Where do we
come from

A study in how we
know anything in

Spoiler Alert:
We don’t !

once upon a time…

What does it all mean?
A Journey to the
Epistemological
Problem of Software
Development

In our Innate Humanness
We optimize for the
probable

Unit Testing

Integration Testing

Happy Path
Engineering

We also optimize
for the possible

Over Engineering

The scaling algo
that never got used…

There is too much to
choose from in the
realm of possible

Actually, we optimize for
the perceived probable

How do we know
what to create?

This is the problem

Epistemological
Problem of Software
Development

We gather data and
rhetoric to support
our theories

There are 3 major
arcs currently in
Software Development

We started as Civil
Engineers

First Arc:
Agile

Agile avoids the
problem

Agile reminds that
we dont know what
we are building

Behavior Driven
Development

BDD = Agile +
feedback

Behavior Driven Development is a
second-generation, outside–in, pull-
based, multiple-stakeholder, multiple-
scale, high-automation, agile
methodology. It describes a cycle of
interactions with well-deﬁned
outputs, resulting in the delivery of
working, tested software that matters.
Dan North , 2009

Amplify
Feedback
Loop

Agile emphasizes
feedback to developers
from their overlords and
sometimes even customers

TLDR;
Rapid Iterations Win

Agile is
our guiding
Light

The world has
changed since Agile

We don’t sell
CD’s anymore

Software as a
Service

The last fifteen years have
brought a complete change in
our delivery cadence,
distribution mechanisms and
revenue models

Second Arc: DevOps

DEVOPS IS THE APPLICATION OF
AGILE METHODOLOGY TO SYSTEM
ADMINISTRATION
- THE PRACTICE OF CLOUD SYSTEM ADMINISTRATION BOOK

Agile
Infrastructure

http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-ﬂickr

Less WIP
Less technical debt

Customers actually using
the feature while the
developer is working on it

Great side effect:
Produces Happy Developers

Devops realized that ops
doesn’t know what devs
know and vice versa

Dev : Ops
10 : 1

DevOps is an Epistemological
breakthrough joining people
around a common problem

Culture is the most
important aspect to devops
succeeding in the enterprise
- Patrick DeBois

Culture is shaped in
part by values

Mutual Understanding
Shared Language
Shared Views
Collaborative Tooling

DEVOPS IS THE INEVITABLE RESULT OF NEEDING
TO DO EFFICIENT OPERATIONS IN A [DISTRIBUTED
COMPUTING AND CLOUD] ENVIRONMENT.
- TOM LIMONCELLI

https://puppetlabs.com/sites/default/ﬁles/2015-state-of-devops-report.pdf

TLDR;
High-performing IT
organizations experience 60X
fewer failures and recover from
failure 168X faster than their
lower-performing peers. They
also deploy 30X more frequently
with 200X shorter lead times.

Culture
Automation
Measurement
Sharing
- @damonedwards, @botchagalupe

Devops gone wrong

“THAT THE WORD #DEVOPS GETS REDUCED
TO TECHNOLOGY IS A MANIFESTATION OF
HOW BADLY WE NEED A CULTURAL SHIFT”
- @PATRICKDEBOIS
http://www.slideshare.net/cm6051/london-devops-31-5-years-of-devops

Third Arc:
Continuous
Delivery

Continuous Delivery is not
merely how often you
deliver but how little
you can deliver at a time

Delivery
Pipelines
are rad!

Batch Size of 1

Separation of Duties
Considered Harmful

Give power to the
Developers to deploy

Reduce Code Latency
Increase Code Velocity

3 Arcs:
Agile
DevOps
Continuous Delivery

The next Arc:
Security
Rugged

“…Those stupid developers”
- Security person

“Security prefers a system
powered off and unplugged”
- Developer

Cultural Unrest
with security in
most organizations

Compliance Driven
Culture

“[RISK ASSESSMENT] INTRODUCES A
DANGEROUS FALLACY: THAT STRUCTURED
INADEQUACY IS ALMOST AS GOOD AS
ADEQUACY AND THAT UNDERFUNDED
SECURITY EFFORTS PLUS RISK
MANAGEMENT ARE ABOUT AS GOOD AS
PROPERLY FUNDED SECURITY WORK”

Security is where ops
was 7 years ago…

Dev : Ops : Sec
100 : 10 : 1

Understaffing means
no one thinks security
helps the business win

DevOps changed that
for Ops, security can
change too

Netflix
demonstrated
that people
care about
resiliency

Innately, we all care

Rugged Software Movement

#ruggeddevops

https://vimeo.com/54250716

http://www.youtube.com/watch?v=jQblKuMuS0Y

Security’s way forward is to
help developers and help
operations

Start there

Let’s review Security’s
approach thus far

BadIdea #1
Applications can’t be
defended—Web App
Firewalls Suck!
lets do developer training

Awareness campaign
OWASP Top Ten

We abandoned knowing
anything useful about
the Runtime

Instead Add Defense
based on behaviors

BadIdea #2
Developers can’t figure it out.
lets scan for vulnerabilities
instead

“here is a 400 page PDF of
our findings to prove your
developers don't get it!”
- The Pen tester

Even with the emphasis
on appsec training, in
practice we made it a
dark art

Integrated rugged
testing should sit
inside the pipeline

BadIdea #3
With the new alignment
to vulnerability scanning,
there is a tendency to Fix
the Low-Hanging Fruit

we still don't know
who is attacking us

We still don't
actually know what
they are attacking

Real Threats go Unknown
so Developers fix what the
automated tooling detected
at a certain point in time

Add Application
Security Telemetry

badidea #4
Put in tooling that no
one outside of security
can understand

usually in the name
of compliance

“Get a Web App Firewall
dude!”
- PCI-DSS Req 6.6

Choose your own
adventure…

smallest possible
solution you can
consider a WAF…

Our CDN added
ModSecurity Ruleset
Huzzah!

An appliance that
blocks all the things

And now you wonder
why no one eats lunch
with you anymore

“every aspect of managing WAFs is an
ongoing process. This is the antithesis
of set it and forget it technology.
That is the real point of this research.
To maximize value from your WAF you
need to go in with everyone’s eyes open
to the effort required to get and keep
the WAF running productively.”
- a whitepaper from a WAF vendor

Ok, Security has to change…
How do we add value
already?

Two ways!

Add value to Devs
Add value to ops

Pray that someone
notices

Pro-Tip #1
Bad-Behavior Driven Development
(automate those security tools!)

Start with Adding just one
test for XSS on a few pages
in your app

gauntlt is Bad-Behavior
Driven Development

GAUNTLT
Open source, MIT License

Gauntlt comes with pre-canned steps that hook
security testing tools

Gauntlt does not install tools

Gauntlt wants to be part of the CI/CD pipeline

Be a good citizen of exit status and stdout/stderr

Be Mean to Your
Code

Gauntlt Uses Cucumber
and its awesome

here’s an XSS attack
Example

@slow @final
Feature: Look for cross site scripting (xss) using arachni
against a URL
Scenario: Using arachni, look for cross site scripting and
verify no issues are found
Given "arachni" is installed
And the following profile:
| name | value |
| url | http://localhost:8008 |
When I launch an "arachni" attack with:
"""
arachni --modules=xss --depth=1 --link-count=10 --auto-
redundant=2 <url>
"""
Then the output should contain "0 issues were detected."

http://theagileadmin.com/2015/06/09/pragmatic-security-and-
rugged-devops/

github.com/gauntlt/gauntlt-demo

github.com/gauntlt/gauntlt-starter-kit

leanpub.com/hands-on-gauntlt
Hands-on Gauntlt
Book

Pro-tip #2
Put security testing in
your continuous
integration system

https://speakerdeck.com/garethr/battle-tested-code-without-the-battle

Pro-Tip #3
Add Application Security
telemetry to devs and ops

Convert App Security
Logs into metrics in the
systems dev and ops use
StatsD

RunTime Correlation
between biz, ops, dev, sec

SQLi Attempts + HTTP 500’s
or
login spikes + transaction
decrease

Runtime
Instrumentation for

Pro-Tip #4
Get hugs from the
auditors and add
Hardening and Audit using
config management

Open Source
Hardening Framework
chef/puppet/ansible
http://hardening.io/

Run Nightly Audits of
your Hardening using
Config Management
(Chef audit mode)
https://www.chef.io/blog/2015/04/09/chef-audit-mode-cis-benchmarks/

OS and Config
Management

reverse the trend
Add Value to Devs
Add Value to Ops

New Farming Methods in the Epistemological Wasteland of Application Security

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to New Farming Methods in the Epistemological Wasteland of Application Security (20)

More from James Wickett (9)

Recently uploaded (20)

New Farming Methods in the Epistemological Wasteland of Application Security