Lean Startup 2012 Simulcast

I am Daniel Greenfeld (blog, twitter), one of the principals and CTO at Cartwheel Web, a software consulting shop that builds startups. I have two startups of my own:

  • Pet Cheatsheets

    Build custom 1-page reference sheets for your pets in minutes.

  • MoveHero

    Get free, anonymous moving estimates.

Eric Ries

  • If you are building a startup you are trying to replace the big companies you dislike. The big companies started the same way you did, as a way to break the current system.
  • We want to build the next big companies by mastering the disciplines of entrepreneurship.

Todd Park - USA CTO

  • https://twitter.com/todd_park
  • Previous - CTO of Health and Human Services
  • Current: - U.S. Chief Technology Officer and Assistant to the President. Tech entrepreneur-in-residence at the White House
  • Worked a medical startup in Boston that went public in 2006
  • Created two more health based companies in Boston


  • Often runs startup-like efforts called ‘entrepreneurs in residence’ inside the USA, which allows radical new approaches for the federal government
  • The FDA has been working with this program to help the health of the nation.

Open Data Initiatives Program

  • URL: http://www.data.gov
  • Ronald Reagan is the godfather of the ‘Open Data Initiatives Program’!
  • GPS grew out of this system which has provided billions of dollars of business
  • The government is the holder of immense archives of useful data


Sunlight Labs (http://sunlightlabs.com/) is a group that works to translate the often not-machine readable data into formats that can be read and used by machines (and hence entrepreneurs).

Todd is working on…


This is why OpenStack (http://www.openstack.org) was started at NASA by Chris Kemp. It was to reduce cost of single server setup from tens of thousands of dollars (mostly labor costs for meetings to discuss setting up single machines) to the same cloud costs paid for Amazon AWS. I can tell you as an ex-NASA employee that server provisioning was overly expensive as of 2010.

Diane Tavenner - Summit Public Schools


  • PROS: Lots of successes! Tons of High School graduates!
  • CONS: Only 50% of their graduates finished their college degrees
  • Something is wrong

MVP concept

  • What if instead of teachers directing classrooms, students went down their own path
  • While this is not a new concept, they decided to map out the requirements extremely clearly to students and parents.
  • They created an on-line testing system so that students could update their status by passing tests so they could see the results instantly.
  • Added tons of testing and metrics. Rather than wait for years for results, they needed to know right away so they could fix it.
  • Refused to use vanity metrics to promote the schools. They needed to know honest, real data on actual results - and kept even the bad news.
  • Encouraged teens to provide feedback through mechanisms that teens like to use.

What they discovered

  • Lectures were not effective.
  • Teachers were much more effective dealing with individual student issues, rather than just broadcasting knowledge.

Tendai Charasika

Get Out of the Building

  • Get out and talk to users
  • Get Uncomfortable
  • Learn quickly and upfront if people actually want/need your idea implemented
  • If you don’t ask you miss out on what they really want.

10 pragmatic ways to get out of the building

  1. Don’t Ask Your Uncle. In other words, don’t ask people you know will say nice things.
  2. Set up a booth, do a public demo
  3. Interview potential customers.
  4. Put your office where your customers are
  5. Throw a party
  6. Talk to experts in the field
  7. Find the decision maker (everyone else is just chaff)
  8. Listen to what customers are demanding
  9. Pre-order, landing pages, analytics (show demand for the product)
  10. Ask for the introduction


Idea: Market your tech startup by sitting in a coffee shop and showing people.

TWO PEOPLE - Eric Ries and Tereza Nemessanyi

  • Tereza Nemessanyi (https://twitter.com/TerezaN)
  • Talking about using general accounting practices.
  • Stay away from vanity metrics, except for what goes into a pitch deck.
  • Investors use vanity metrics to make investments in your project, but using them for concrete business decision making is dangerous.
  • Issue: Investors often use your original vanity metrics when determining how well your project is doing

Beth Comstock interviewed by Eric Ries

  • Beth is the Chief Marketing Officer of General Electric (GE). https://en.wikipedia.org/wiki/Beth_Comstock
  • GE believes that entrepreneurs are everywhere
  • GE has to keep reinventing itself: You don’t last for over 130 years by staying static.
  • Lessons learned: Partner with outside firms to help bring outside ideas into the company
  • Recently: GE got into energy storage (batteries) via startup/entrepreneurs and it is now a multi-billion dollar part of their business
  • Lesson learned: Really focus on MVP before trying to make it perfect for market. This is critical before ramping up to large production efforts - otherwise you have no idea what the problems really are.
  • Statement: You can fall in love with your technology or you can fall in love with what your customers think about your technology.
  • They want help and will pay for it! See http://www.gequest.com

Jessica Scorpio

Were not sure if it would work, so began prototyping.


  • Worked with students out of Moffet field, near San Francisco, to see if it would work.
  • Built an iPhone app right away to get them a working prototype.
  • Competed in Tech Crunch Disrupt to get publicity and won.
  • They have a custom product called CarKit to let it wire into your car.
  • Worked in litigation because part of this means granting easier access to your car. What if someone else is driving it and wrecks it? By getting some laws passed in California they cleared up the rules for making this service work.

Daniel Kim


  • Building a car is hard.

  • Building a car and mass producing a car is crazy hard.

  • If you are creating a car company, you should know how to build a car, not just be a car executive.

  • Trying to build the Model-T of the 21st century. Getting it right means positive income for 90-100 years.

  • Different approach from segway

    • Spent a lot of money doing research if there was a need for a small, sustainable vehicle market.
    • Did building of product after doing market research
  • Engineering:

    • Built by hand, rather via expensive machinery.
    • Didn’t worry to much about meeting prototype deadlines
  • Feedback

    • Did a small production round to demonstrate that people would buy it. This impressed investors
    • Got lots of feedback from users and drivers

Lane Halley

Process for building products

  • Sketch out your ideas as a team

    • Lowest response fidelity
    • Cross functional pairing is important
  • When designers and developers work together, they need to understand each other’s tools.

  • Lean startup is great for design

    • Quick
    • Visual
    • Collaborative
    • innovative
  • Use workflow sketches to determine the flow of a product

    • Don’t worry if it’s ugly, use paper
    • Don’t use fancy tools
    • If you use fancy tools, you risk locking up your product in whoever controls the fancy tools.
  • Wireframes

    • balsamiq is great
    • So is paper

Ron Williams

Kind-of-lean startup talk

  • Founder of Knodes

  • build/measure/learn for everything… or else

    • Build: If you don’t build it you don’t know if it can be done
    • Measure: Find out how it’s used, by people or whatever
    • Learn from what you observe.
  • Telling your team to BE lean is like a crash diet

    • Don’t say: Hey I just read this awesome book and we’re going to start doing these 15 things differently.
    • Changing habits is HARD.
  • Beeing lean isn’t your goal

    • The real goal is to have fun creating a product your customers love.
    • GitHub is a GREAT example.

Andres Glusman

RSVPs are going up? Here is why:

Myth: People give a damn about lean methodologies

  • No one wants to switch gears

  • No one wants to buy a process

  • Instead of convincing, just start doing it.

  • Avoid Malkovich Bias

    • The tendency that everyone uses technology the same way that you do.
    • Example: iPhone/iPad users often don’t realize that the Android market is larger than the iPhone/iPad market.

Myth: People want to test things

  • People actually like to build things

  • Because of this issue, try to test easy things.

  • As you improve your system thanks to easy test results, testing becomes more exciting

  • Failure:

    • Don’t try to avoid failure, embrace it.
    • Learn from each mistake via metrics and tests and improve ever since.
  • Go after the things that will cause us to fail as fast and often as we can.

Reality: People want to build and test things.

Myth: You can test your way into a great experience

  • Testing your way to an experience often means you create a complete and total mess
  • Sometimes you have to restart from scratch and see how it goes.
  • See http://www.meetup.com/create/ to see what they’ve managed to get working

Panel - Getting engineers to embrace Lean

How to get developers/engineers to switch from Agile to Lean.

  • Get engineers to embrace smaller prototypes
  • Get your engineers to embrace metrics
  • Throw away the code when you are done with the MVP


Read the ‘Danger: MVPs often not disposable’ section below.

Danger: MVPs often not disposable

Throw away code after the MVP is done? That only works for established companies.

Anyone who thinks you can throw away MVP code hasn’t talked to anyone at Twitter, GitHub, or 95% of other companies. They still run off the original MVP code. The only companies who can get away with throwing away MVP code are pre-existing companies with multi-million dollar budgets who use MVP efforts in tiny segments of their system architecture.

TWO SPEAKERS - Nikhil Arora and Alejandro Velez

Stephanie Yeager

@ http://twitter.com/steph_hay

Using words that help people find you and choose you

  • You want people to choose you.

  • But words describing superlative are overused. Everyone is ‘the best’.

  • Try using ‘Lean content’ to describe your product to someone who isn’t you.

  • Look for the ah-ha body language

    • See the questions they respond to you with before the ah-ha moment
  • Use the mom test. If you feel uncomfortable explaining it to your mom, then you need to find a better way.

  • Growth goal: Get found

    • Test your messages in AdWords. Test for clicks, not conversions.
    • Embrace the unsexy words in organic searches
    • Look for Entry Points and Top Content in GA

Steve Blank


Teaching Entrepreneurship

What we used to believe: Entrepreneurship can’t be taught.

What we know now: Entrepreneurship can be taught to anyone who volunteers to try.

Learn Entrepreneurship

What we used to believe: Learn Entrepreneurship requires a lot of education

What we know now: Learning Entrepreneurship some theory and a lot of practice


Learning entrepreneurship from an educator is risky. Their experiences may not translate to today’s conditions.

Teach the Entrepreneurial API

  1. Teach how to create a business model canvas
  2. Teach understanding of Customer Development
  3. Teach how to implement the plan using Agile Engineering

George Bilbrey

“Enterprise in the lean startup”

  • Part of Return Path: http://www.returnpath.com/

    • new product: Anti-phishing system
  • Built with small team inside of their large 400 person company

  • Read all the lean books

Lessons learned

  • Determine who the buyers really are.
  • Bring in a Salesperson earlier in the process, however, the salesperson must like experimentation.
  • Prepare to pivot: That means you have to be ready to admit you got it wrong
  • Start small and organize for experimentation.

Ivory Madison

“Bonfire of the Vanity Metrics”

  • Vanity blinds you to a lack of actually important data
  • Mark Twain: “Facts are stubborn, statistics are more pliable.”

Don’t use these metrics

  • Page views
  • New members
  • Total members
  • conversion rate
  • Percent growth
  • Twitter followers
  • Facebook friends or likes

Characteristics of actionable metrics

  • Measure success at your core business
  • Show direct relations to revenue

Your Four: Most important Metrics

  • Measure revenue
  • Measure Sales Volume
  • Measure Customer Retention
  • Measure Relevant Growth

Find the big picture in???


They switched back to the speaker after 2 seconds. :P

Ash Maura

“Getting the ultimate metrics dashboard”

  1. Establish a standard measure of progress
  2. DaveMcClure’s Pirate Metrics (look them up)
  3. As you gain users, it becomes harder to measure progress.

Leah Busque

  • Founder and CEO of Task Rabbit

“If you had only $1, where should you spend it?”

  • Really understand your customer so you can target your acquisition techniques

  • Be holistic:

    • test everything
    • not just channels
    • not just funnels
  • Geo-targeting is critically important.

    • What works in one place will not work somewhere else
    • Test and measure the results

Big Panel

  • Scott Cook (Intuit boss)
  • Carol Howe
  • Joe Hernandez
  • Barath Kadaba (VP of engineering)

Question: What is the goal you have for your venture?

  • You want to stay small and insignificant? (0%)
  • You want to be giant and well known? (100%)

Making it happen

  • Scott:

    • leaders need to change and lead this change into the business
    • Change things to create success after new success
    • Large companies typically get stuck and become stifling
    • Companies lean on politics and slide desk to stop changes:
    • Leaders need to stop deciding on opinion, but to work on actionable metrics

Components of making it happen

1. Leader has to set the grand challenge

Barath Kadaba

  • In 2008 he was told to change the lives of India. All the lives

  • Given budget for just 3 people to do it.

  • First effort:

    • Decided to focus on the lives of Indian farmers.

      • 150m+ of them
      • Contribute 25% of India’s GDP
      • Most live in poverty
    • Decided to solve the narrow problem:

      • Problem: To whom can they sell their produce to get the best price?
      • Solution: Send farmer’s SMS text messages with the latest known data
      • Quick Implementation: Faked it with hand-texted SMS messages to farmers.
      • HUGE success
  • They got 20+ projects done this way

    • Team fought management death threats to stay alive
    • Only survived because they were so small
    • Yet increased the income of millions of farmers by 20%

2. Leader has to implement organization settings to make it possible to change

“Lawyers often are the barrier to success, they need to be instead considering how to make success more possible”

Joe Hernandez

  1. Change Mindset, which will change Behavior.

    How do you shift a group from saying no (leaders, lawyers) to saying yes?

  2. Democratize Action

    Create a clear set of guidelines in non-legalese that makes it easy for people to understand when they can move forward.


    How is a set of guidelines ‘democratization’?!? I think he needs a dictionary. :P

  3. Becomes the power of success

    Enable easily understandable rewards so you can demonstrate success. Payment can be financial or simple numbers.

3. Leader has to model pulling insights from both successes and failures

Carol Howe

  • In 2009 created a start-to-finish app for Intuit that lets you take pictures of your tax documents and it files for you.

  • But this wasn’t how it started:

    • Started with a photo capture app that would upload to your computer and that would file to the government

    • But when they created the app prototype, testers made it clear they wanted to just finish it on their phone

    • Stepped back and looked at the feedback from prototype users and listened carefully

      • Mobile fans raved in long discussions
      • Web fans said one word answers like, “nice” and “neat”
    • Started with launch in California and took lessons from there

4. Leader has to live by the same rules and disciplines as everyone else.

Scott Cook

  • Test your beliefs the same way you make others test theirs.
  • If you don’t test your beliefs, then you’ll drive into places based on opinion, not science.
  • By testing your hypothesis, you don’t just get better results, you often have more fun.

Drew Houston

  • Founder of dropbox.com

Q&A from questions given from the audience:

  • Question: What do you look at in regards to metrics?

    • Answer: We look primarily for: “How many active users do we have?”
  • Question: What tools do you use for gathering metrics?

    • Answer: The simplest tools possible to gather metrics
    • Answer: Store them in google docs and other simple tools
  • Question: How do you find people?

    • Answer: Personal network
    • Answer: Connect with the developer/business communities
  • Question: What are your goals and how are you accomplishing it?

    • Answer: Build something that makes me happy
    • Answer: Build something that makes others happy
    • Answer: Have fun making it work
    • Answer: Figure out how many users you need to get in order to do the startup full time.

Charles Hudson

“Being a VC does not protect you from making boneheaded mistakes as founder.”

How they got started

  • Saw that none of the games for Android were any good

  • Decided to become the ‘android guys’

  • Platform decisions matter

    • Tech
    • Distribution Channels
    • Can they go between systems?

Had to pivot

  • Couldn’t monetize just on Android
  • Tried to leverage switching to Kindle Fire and iOS
  • Story isn’t done yet

Dave Binetti

When you do you pivot?

  • You need to have a vision to make a decision based off of hard metrics.

    • Often people make a pivot based not on hard metrics but emotion
    • Pivoting doesn’t mean changing your vision, it means changing your path

Mark Abramson

Did Lean Startup Machine and won it. Ran 10 experiments and pivoted 5 times during the conference

  1. Experiment - Tax paying

    • Discovered that restaurants over 25 employees have to pay an extra tax
    • They all send it accountants and pay serious money.
    • No pain. Not worth doing.
  2. Experiment - Happy Hour Marketing

    • No one has problems here. Not worth doing.
  3. Experiment - Getting people into food places

    • Fierce competition everywhere. Not worth doing.
  4. Missed

  5. Experiment - Wine club for restaurants

    • 6 bi-monthly events in 2013 for the serious wino with exclusive chefs

    • $1500 for 2 people for an annual membership

    • People will pay for this service!

      • They’ve made $4500 in days
      • They could have sold out if not presenting

Marc Andreessen

Interviewed by Eric Ries. Notable quotes:

  • Pivoted twice when it was still called, “We fucked up”
  • When you get a ton of customer service requests it means you are succeeded.
  • You have to move quickly in order to capture the market. You can’t wait. Just have to move.
  • Until your effort makes a product market fit, it’s not a real company.

We learned a lot of lessons from the dot-com crash

  • Worried that people who lived through the crash are suffering psychological damage from the event.
  • Many of the ideas of the time were valid, but were just too early.
  • You can take the ideas of the time and with a twist, apply them to great success
  • The bubble itself was only 18 months. From 3rd quarter of 1998 to the 1st quarter of 2000.

Problems he hears in pitches

  • Not every startup can startup can be lean. Sometimes you need to just be audacious. Think Apple, Intel, or anything Elon Musk does these days.

    • You can’t put a rocket into space on a lean program.
    • Don’t let the lean startup methodology destroy other ways of doing things.
  • Lean startup methodology used to avoid sales marketing strategy seriously.

    • Sales and marketing doesn’t happen magically.
    • No matter how good your product might be, people won’t come to it without sales and marketing.
    • Critical examples: Google, SalesForce, Facebook, and Twitter have thousands of sales rep.
    • Business applications do not sell themselves
    • Don’t rely on anything going viral
  • Entrepreneurs give up too quickly

    • Are you going to do the heavy lifting over a long period of time?
    • keep at it!
  • Failure fetish

    • By taking the stigma out of pivoting, entrepreneurs have an excuse to not try hard enough. Don’t be gleeful about failure.
    • People who claim to be “serial entrepreneurs” without success are giving themselves a fancy title for being a failure.
    • Preserve the good of failing, learn from it and succeed next time.
  • History is weird

    • Winners are portrayed perfectly, losers are portrayed like idiots
    • He contends that winners often are just lucky enough to start at the right time.


  • Do not go public until your company has built a fortress. If you don’t have all the positions filled, brand established, predictability in the market, then you are at great risk.

  • Going public today is an extreme sport. It’s very dangerous.

  • Lean Startup is like the Theory of Relativity for Business.

    • We now have a process and science for getting things done.
    • BUT you still need old fashioned sales and marketing.