• Talk title:Keynote: Why Does Bad Software Happen To Good People?
• Matt Tesauro (@matt_tesauro)

Matt Tesauro was the project lead for the LiveCD OWASP Project and is on the OWASP board. He gave the LASCON keynote address, video shown below:

LASCON 2010 - Matt Tesauro - Keynote from Josh Sokol on Vimeo.

Here are my notes/tweets from the presentation:

• The historical perspective of OWASP and application security:
  1. Software is everywhere (Ex: new Barbie dolls)
  2. Software has lots of problems:
     • Example: Air France 447 in 2009 (Brazil to Paris, disappeared). We spent 2 years searching for black box before finally finding it
     • Jaguar recalls 18K cars, over cruise control software not turning off ("I suppose I'd rather die in a Jaguar than Toyota Prius")
  3. OWASP creates visibility: "allows us to break the cycle of find, hide, blame" between appsec community, vendors and users
     • Jeff William's vision of ideal: "create equivalent of the nutrition label on foods," but for software
• "Goal: fail often in order to succeed sooner" vs. "worshipping at the alter of success"
  • "I have not failed. I just found 10K ways that won't work" --Thomas Edison, on lightbulb invention
  • Alan Mulally, Ford in 2006: asked stat reports (red/yellow/green); all execs reported green, despite losing billions
  • Citing Productive vs. Unproductive Failures: Amy C. Edmondson: Harvard Business Review
• Edmondson: noted that people more candid/willing to criticize rough prototypes vs. high quality prototype
  • (Opposite of last minute user-acceptance test, right before production deploy. Haha.)
• Lessons from Failure: Deepwater Horizon Oilrig, Fukushima Nuclear Plant; what can we learn?
  • Lesson: growth isn't linear or smooth; not like balloon; more like sea urchin: don't know what vectors will hit
  • When that sea urchin has broken spine, tech to fix the problem is not likely to be mature: relies on improvisition
• Improvisation at Fukushima/Deepwater was costly, resulted in widespread damage; b/c tech pushed so far to edge
  1. 'Even if things seem safe/secure day in/out, disasters will happen." (blowout preventer, partial meltdown)
  2. Develop some broadly applic technology for mitigation before needed (e.g., API disappears, load balancers crash)
• (Anti-pattern: freak out at XSS risk, then deploy WAF: "wrong thinking; you have it backwards")
• Universal force: "self-regulate or accept government regulation" (citing PCI DSS precedent)
  • Safety case regulation requires relatively long term relationships (to gain mastery, expertise, relationships)
  • (Interesting that average PCI DSS assessment crammed into 4 days: how much mastery can be gained in 4 days?)
• "How not to react: Patrick Webster & Australian retirement fund:" found security flaw in site: incr ID in URL by 1
  • Upon reporting security flaw, polic showed up at his house, confiscated all computers, lawyers demanding $$ to fix
  • End result: "press went wild, and tons of 'pro-bono' testers started 'tested' and posting"
• Way to do this right: Bug Bounties + common sense: e.g., Google, Mozilla
• Questions: at design time, is there talk of future testing?
• Questions: does system allow for testing w/o Herculean effort. (Nice)
• "Don't buy fail": use procurement process to block fail: OWASP Legal Project, OWASP ASVS
• "Why OWASP Will Win: We have an awesome community"
  • Study by econs at MIT, Univ of Chicago, Carnegie Mellon: econ rewards lead to worse performance
  • Why? Knowledge workers want autonomy, mastery, purpose. (This is Dan Pink's work. cc @timgrahl)
  • Dan Pink's book called "Drive: The Surprising Truth Of What Motivates Us"
  • #lascon: @matt_tesauro awesome call to action: "want autonomy, master and purpose? Work with OWASP". Nice,.

• Talk title:A Statistical Journey through the Web Application Security Landscape
• Jeremiah Grossman (@jeremiahg, personal website, slideshare)

Jeremiah Grossman is the founder of White Hat Security, where my good friend Stephanie Fohn is currently CEO (she helped us with our first initiatives and product launches at Tripwire over a decade ago, for which I'll be forever grateful). Jeremiah is also very well-known for his work on metrics and benchmarking all aspects of vulnerabilities.

Here are my notes/tweets from Jeremiah's presentation:

• "Statistical analysis of web application security landscape": "he & daughter flew 4K miles to be here"
• "For last 10 yrs, I've been doing nothing but web app security; it's important, as 5 of 7 most valuable companies are technology companies [the remaining two companies being in the oil extraction business]"
  • "And for that matter, even oil companies are technology driven"
• "My passion has become how breaches happen. My hope is that we can trace that backwards to prevent."
• "The VzW Data Breach Report continues to be a must-read report"
  • "in 2009, 95% breaches were remote org criminal grps hacking svrs & apps"
  • "in 2011: web application breaches increased last year, made up nearly 40% of overall attacks."
  • "...and those stats were before countless big breaches: Sony, NASDAQ, Gawker, etc..."
• "What we should learn:"
  1. these breach could happen to anyone.
  2. even one vulnerability is enough to result in significant disruption business."
  3. Attack techniques of choice: SQL injection, PHP local file include, password reuse, DoS, malware" not sophisticated * (See any @hdmoore presentation to get a sense of of lethal use of employment of password reuse to p0wn)
     • "First citation of SQL Injection attack: Christmas 1998. It's a 13 year old threat"
  4. "What makes new breaches unique is the relentless of the attacker; I would argue that Sony was not materially worse than average".
• Q: (Someone asked about the recent BofA outage, suspected to be a security breach) A: Here's a news article on the BofA operational/deployment probs
• "Avg annual amt of serious vulns in yr 2007-2011: 1111, 795, 480, 230, 148" ('serious': potential breach/data loss)
  • "We're seeing a downward trend: introducing 10-11 vulns/month; Majority in retail, financial svc, telecomm"
  • "Financial services companies do significantly worse than banking." (Whoa! ASPs do worse than banks!)
  • "Worst industry is Retail, which is most affected by PCI. Don't see security effect of PCI DSS"
  • Note: Very interesting: Retail sector also has very low deploy rates, compared to FiServ/Banking. That doesn't match my own experience...
• WhiteHat Security Top 10: Info leakage 64%, cross-site scripting 64%, content spoofing 43%; cross-site forgery 24%
  • (Dude, I would love to help u do data visualization like Dr. Hans Rosling, to show temporal axis :) Like this
  • 50% of banking will fix in 30 days. Challenge is remediation; (Obvious, right?)
  • "We see steady improvement in resolved vulnerability: incr at 5% year over year"
• "Here is the big question: Why do vulns go unfixed?
  1. No one responsible/owns code
  2. Dev group doesn't understand/respect the vuln;
  3. Lack of budget
  4. Code owned by vendor;
  5. Decommissioning soon
  6. Risk of exploitation acceptable
  7. Vuln conflicts w/biz use case
  8. Compliance doesn't require fix
  9. Feature enhancement are prioritized ahead of security fixes" (<--- surprised this isn't #1 or #2)
• "Testing speed & frequency matters; how quickly you can vuln data to developers; within 1 wk -> less than 1h to fix"
  • "Dev learns in 1month: 1-3 h to fix; 1 year: more than 10h" (b/c of retasking, relearning, etc. Continuous testing key)
• "Why do breaches happen? Policy. Showing IT Budget game: ask CFO to divvy budget to Apps, Host, Network"
  • Theory: Allocation of CFO vs. CISO budget are 180 deg out of phase: http://t.co/5TOkaqay
  • Making case that CSOs prioritize network, host, then app: mismatch to actual risk.
• Ideas on remedying this results in difficult choices
  1. reallocate resources away from firwalls, IDS, AV towards app security"
  2. Justify brand-new app security spending (difficult in this econ)
  3. Let breaches continue to happen"

• Talk title: Games: The New Science Of Engagement (Gamification),
• Gabe Zichermann (@gzicherm, personal website, slideshare)

This was a fantastic talk. Gabe Zichermann helped codify the gamification, writing a number of books on the topic, including "Game Based Marketing: Inspire Customer Loyalty Through Rewards, Challenges and Contests" and also the O'Reilly book "Gamification On Design".

Slideshare is here:

The tweets I posted are below:

• He spent 8K hours playing Civilization; "I've met the creator of Civilization, and it was awesome"

  • "I was educated by games by 8K hrs of Civilization, equiv to 4yrs of work; Where in World Is Carmen San Diego"

• "Here's an exercise: think about a hypothetical coffee shop, where you're waiting in line: which would you choose? Choice A: you'd get whatever coffee you ever desired and it would be instantly served when you ordered, or Choice B: you'd get bypass the entire line"

  • "In most exercises, Choice B usually wins"
  • "Economists: your time isn't that valuable, esp if you're salaried: 3m you save in line can't used productively"
  • "Rational people [as defined by economists] would choose A. But in actuality, none of us are rational; Humans aren't rational animals"
  • "Gamification says it's okay to design choices for irrational people for good experiences"

• "We are irrational -- we make choices in split seconds, and our rational minds backfill and justify our decisions to ourselves"

  • "Gamification allows us to create systems that are aligned with our own internal intrinsic processes"

• "Here's an idea for game: plot of land, till soil, plant seeds, wait plants to grow, sell it; does it sound like fun? No. It's like serfdom!"

  • "Farmville shows that mechanics are fun, not the theme;"

• "although change appears to be difficult, many things chg very dramatically: Moore's law, CO2 count, .."

  • "These are examples of change that comes out of nowhere: exponential (collective) vs. linear (indiv)"

• "Gamification foundation: desire to mastery: intermed steps: incentive, challenge, achievement, feedback"

  • "Can't learn German overnight: requires social feedback: which is core of almost any good game"
  • "Games are intrinsically wired to produce pleasure: through dopamine: the wonder drug"
  • "Merely the effort to learn a game [eg, juggling] causes pleasure and 15% increase in brain grey matter
  • "Games may increase 'fluid intelligence' [vs crystalline]: novelty, creativity, do things hard way, network"

• "Game challenge tends to increase exponentially: [showing levels of Plants vs Zombies]"

  • "In my generation: all we had to do was jiggle joystick, press button; current gamers req 6 skills before 2yo"
  • "Alternate explanation of current ADHD: 'current world is too slow' [that's why people tweet: haha]"

• "Gamification: is the process of using game thinking/mechanics to engage audiences/users"

  • "It's not just throwing badges up on your website: like FourSquare, etc.; if no desire, gamification won't work"
  • "Gamfication can ignite desire, like throwing gasoline on fire"

• "Narrative for bandages or soup companies doesn't need games, shooting zombies"

• "Designer Kevin Richardson rethought automated speed cameras: fine based on income: $250K speeding tkt poss."
  • "Portland is one of the rare places that examples gets applause." Haha.
  • "Richardson: all ppl driving past camera entered into sweepstakes for speeding fine winnings." (wow)
  • "20% reduction in speeding violations. Unprecedented reduction!"
  • "Imagine ramifications to US tax code; think about incentives [or lack of] for filing on-time"
• "Showing Target POS system: scores based on how long sales transaction should take: immediate feedback to cashier"
  • "Turnover is down, speed is up, customer satisfaction is up. Gamification incr feeling of personal control"
• "Rypple/DueProps: I've done my time in khaki-land/corporations: annual reviews suck. no immed feedback"
  • "Empowers ppl to give positive feedback to all employees, all the time; instant feedback, reduce cycle time"
  • "Companies like Facebook are using Rypple; StackOverflow is another gamified experience. Karma principle"
• "Problem: Altruistic actions can be systemized; monetary reward is counterproductive; Ex: helping elderly after they fall"
  • "Maybe I can trip old people & get $5 for helping them; changes altruistic behavior"
  • "Notion: karma points given that don't turn into $$; just makes u feel good"
• "Charlie Kim runs Nextjump; wants employees to work out/fitness; put gym in every office; 3-4% worked out"
  • "Put out cash reward, split $20K/yr; got 12% participation; next: team based play & leaderboard"
  • "Now 75% participation: team based & leaderboard; not even gyms have that high participation!"

• Another example application of gamification: Google travel expense policy: "Google wants to cut travel expenses & reduce carbon footprint; policy in travel system; easy to game"

  • "Can upgrade future trip, donate to charity, or get 50% cash back at end; has saved millions $$..."
  • "Incentive; publish prices of city pairs & per diems; any $$ under per diem is banked"
  • "But more importantly, people talk endlessly about how to save travel costs; Amazing results"

• "Other examples: hybrid vehicles: game in dashboard to motivate eff fuel usage; Fusion has Tomaguchi-like pet"

  • "Instead of stats, you have to take care of virtual pet; best is Nissan Leaf: compete vs friends"
  • "These car tools are $100MM investment; why? reinforce daily why you bought hybrid car!"
  • "Now, for 1st time, car dashboard is viral; even passengers are engaged; reinforces brand"
• "Book: Decoded by Jay-Z; target not trad biggest book buyers; so gamified: 1st person to find 300 pgs won prize"
  • "Prize: 2 tickets for any Jay-Z event, for life. Amazing price: didn't cost Jay-Z anything. Valuable to fans"
  • "Took 30d for someone to win; winner hired ppl to find all the pages [unanticipated gaming of system]"
• "Progression to Mastery: good example: WeightWatchers: badges, points; changes u in way only gamification can"
• "Bartle's Player Types: Killers, Achievers, Explorers, Socializers" [axis: ppl, acting, env, interact]
  • "Achiever: like Tokyo Fish Market: $1B fish sold daily, the old way: person standing in market"
    • "Most ppl aren't achievers: 'my goal: take kids to school, not get fired, watch 2 1/2 Men."
    • "World isn't designed so that everyone can win: achievers like it: want authentic wins"
    • "John Houston: working w/hypercompetitive; will find ways to compete even w/o incentives"
  • "Explorers: the story of how they do something worth more than monetary value: like finding all keys in Mario Bros"
  • "Socializers: [showing scene from Valley Girls]: scene is not about shopping, but socializing"
    • "Socializers: [showing scene from Valley Girls]: scene is not about shopping, but socializing"
    • "Bridge, mahjong, poker: all context for amazing social interactions; players will talk all nt long"
  • "No product has failed in history that creates authentic social interactions; Socializers are majority of ppl"
  • "Historically, most games have been social; 1974-present is abherration: audience not young males"
  • "Killers: [showing clips of fave shows: Extreme Couponing & Storage Wars]
    • "[on Storage Wars]: Killers want to kick ppl's ass, want to have everyone see it; must put them on rails"
    • "Killer goals often diametric opposite of community; killers win, community loses"
• "SAPS Principle: Status, access, power, stuff: in order of desire, and expense to community"
  • Status
    • "leveling up: even among anarchists, bike messengers; ppl derive status from how anti-establishment they r"
    • "3 or 4 domains of status: church, politics, industry; now myriads 100K status systems to climb"
    • "A guild leader in World of Warcraft as meaningful as deacon of somone's church: higher status = health"
    • "Higher status people are less stressed out, live longer, healthier"
  • Access
    • "like Jay-Z prize; Gilt Group flash sale site: top 0.1% shopper get Gilt Noir loyalty program"
    • "Gilt Noir gives you 15m earlier access: not free stuff; means you'll spend more $$; costs nothing to give"
  • "Power: editor in Wikipedia; guild in WoW; moderator in forum; power benefits are doing unpaid job"
    • "Wikipedia editor takes 20h/week; power to accept/decline articles is power"
  • "Stuff: don't people really want discounts & free stuff? not really; thot exp: free coffee or telepathic barrista"

• Q: "what would gamification suggest on the 1% and 99% wealth distribution?" * A: "no game designer assumes world is self-regulating. Killers ensure it"

  • "Casinos assume they're under attack constantly; has emergency stop button to stop game, to allow investigation"
  • "In any complex system of value, game designers must control the system; Securities regulation has lots to learn"

• Q: "Do any of these concepts apply in education?"

  • A: "Yes! Mr. Ananth Pai in Bear Lake, MN; did experiment where in 18 weeks, using team and gamfication, went from 3rd grade to 4th grade reading level. 'It's fun and social'"
  • Here's a link to Gabe's blog entry on this topic: there is a reference about a TEDx talk he did on on this, as well:
  • "Those games were both motivational; Lesson: must be fun first, and educational second"
  • "Parents/teachers; must not constrain experiences by what you think us; must put yourself in kids' shoes"

• Q: "Tell me about Folded, where they used games to solve protein folding game"

  • A: "Folded was game built to solve HIV"
  • "Eluded researchers for 20 years; mass labor game: 3K ppl played at peak; ppl good at it, algorithms bad at it"
  • "It's a game designed to get work out of people: the CAPTCHA is designed to make ppl work for outcomes"

Why I joined the BSides board

Over the years, I’ve come to respect the work of everyone in the BSides community.  I’m amazed and continually reminded of how many people BSides has positively influenced and the vibrant community they’ve created.  I’ve been to three events, and during my tenure at Tripwire, we became one of the first global sponsors.  I’ve always loved the people who congregate there, and I’m grateful for how it's reconnected me with old colleagues and friends.  I proudly consider myself a part of the BSides community.

In mid-December, I was asked by Mike Dahn and Jack Daniel to join them on the BSides board.  I first admitted to them that I’ve primarily been a beneficiary of everyone’s hard work, and that there are countless people who have contributed far more than me.  But after talking with them, I told them that it was a privilege to be asked and that I would be happy to serve for a one-year term and help in any way I could.  

My goals

My goal is to help ensure that BSides succeeds in its mission: to continue to help more information security practitioners achieve their fullest potential, both now and in the future.

Clearly there have been some growing pains. To paraphrase Bill Brenner, this is really an opportunity to "make a better BSides." Our goal as a board is to help BSides grow, become more effective and accountable, as well as more transparent.

A brief status report

Mike, Jack and I started having nearly daily, now weekly, phone calls. The top issues we're working on are the following:

• Create a timeline to complete all the filings necessary for BSides to become officially a 501c3 not for profit corporation
• Create a timeline to retain an outside bookkeeper and release audited financials, going all the way back to the first events, to show that all account balances and values are exactly as they should be, and that all the money went where it was supposed to
• Create a communication calendar so that we regularly release information on what we’ve promised and how we’re doing on those promises, in order to earn back any lost trust with the community

On the 501c3 front, the team continues to move towards the official filing. BSides remains a California public benefit S corporation. As such, there is no official board, but we’ve started to organize and adopt all the structures and processes required for when we have official 501c3 status.

Part of this is getting regular financial reports released that are audited by an independent third party. The team has spent two weeks interviewing firms to take over the daily bookkeeping operations, as well as a CPA firm who can attest to the accuracy of the financials.   I’m particularly pleased that as soon as BSides completes the transition to a 501c3, the CPA firm that opines on the financials of the widely-revered Electronic Frontier Foundation (EFF) will do the same for BSides.  

While I've studied the SecurityErrata post, based on my analysis, I believe that any financial reporting errors found will be small. My biggest concern is that volunteers sometimes paid event suppliers out of their own pockets, due to BSides cash flow issues -- these transactions may not have been recorded or repaid properly. Of course, we will fix any issues we find.

And finally, on the communications front, this will be the first of many communications you’ll see from the team to make you aware of what we’re focused on, and how we’re doing on the commitments we’ve made.

Some last thoughts

Mike and Jack have been terrific to work with, and I’m confident that we’ll have more positive information to share throughout January and February.  From there, the focus of the board will be to discuss the structure that will best serve the BSides mission and community.  

I want to thank the many people who took the time to give me advice, provide recommendations on trusted bookkeepers and accountants, and much more. I particularly want to acknowledge Branden Williams, Brian Costello, Matt Hixson, Todd Butson and Bob McCarthy and countless others for their help, for which I’m very grateful.

I can't say enough good things about this conference.  If you want to know what the bleeding edge of Web Operations and Development execution look like, this is a conference you can't afford not to be at.  Whether you're Ops, Dev or Infosec.

I had the privilege of presenting on Wednesday.  The talk title was "Creating the Dev/Test/PM/Ops Supertribe: From Visible Ops To DevOps." It summarized some of my key learnings since co-authoring the Visible Ops Handbook and studying high performing IT operations organization, and why I'm so excited about the DevOps movement.

Jesse Robbins (@jesserobbins), John Willis (@botchagalupe) and Patrick DeBois (@patrickdebois) convinced me to talk about some of the projects I'm working now, which I describe in the presentation, as well.

Slideshare is below, followed by the talk abstract.

I’m going to share my top lessons of how great IT organizations simultaneously deliver stellar service levels and fast flow of new features into production. It requires creating a “super-tribe”, where development, test, IT operations and information security genuinely work together to solve business objectives as opposed to throwing each under the bus.

I will describe what successful transformations look like, and how they were achieved from a Dev and Ops perspective. It will draw upon my 11 year study of high performing IT organizations, as well as work I¹ve done since 2008 to help some of that largest Internet companies increase feature flow and production stability.

Lastly, I will share materials from two book projects I am currently working on: “When IT Fails: The Novel” and a prescriptive DevOps guide. I am seeking fellow travelers who want to capture and codify the best known methods, recipes and case studies of how to implement successful DevOps-style transformations.

In this post, I'm display the tweetstream from the talk.

Now, normally, I'm the person generating the tweetstream -- I like taking notes of the proceedings for topics I'm interested in.  And at conferences like #SxSW, it's incredible to be part of a lively backchannel.  When there's a group of attentive people tweeting, you can almost get the gestalt of the talk, just by viewing the tweets.

I'll be posting more on the research that William Hertling (@hertling) and I have on this in a future post.

But in the meantime, here's the Twitterstream for our #hackthemachine talk.

PS:  It is surprisingly difficult to copy/paste a tweetstream into a blog.  I had to write a Ruby app to do this.  Anyone interested in seeing how I did this?