Keynotes¶
Python in the Web: Can we keep up?¶
By Audrey Roy
I’m partial but I think she did a great job of pointing out where Python might/should go.
Metaprogramming¶
By Jeff Rush
“Metaprogramming is the programming of programming”
Note
Apologies, but I couldn’t keep up with the firehouse of knowledge. This is woefully incomplete.
Warning
This is from me: Metaprogramming should be done carefully. Only use it when you have no other options!
This talk makes use of¶
- metaclasses
- decorators
- attributes
- more
What is metaprogramming orientation¶
Code Manipulates Data
Data Feeds into Code
Metaprogramming sits about the Code/Data
Add/adjust elements
register elements
tag elements
event management
- Pre/post initialization
- read/write
- call-and-return
Addressing a problem¶
class Request(object):
class HTTPServer(object):
def handle_request(self, ...):
req = Request(...)
objective
- subclass a class deep inside a module
- Requires rearrangement content of the module
old_import = sys.modules['__builtin__']
sys.models['__builtin__'].__import__ = self.__import__
Sample code¶
class Member(object):
__metaclass__ = DatabaseTable
dbtable = "Members"
class DatabaseTable(type):
def __init__(cls, name, bases, class_dict):
col_defs = db.query_cols()
for col_def in col_defs:
db_column = wrap_col_rdwr(col_def)
setattr(cls, col_def.name, db_column)
def wrap_col_rdwr(col_def):
def get_dbcol_value(self):
return self.__dict__.get(col_def.name, None)
def set_dbcol_value(self, value):
value = col_def.validate(value)
self._-dict__[col_def.name] = value
return property(get_dbcol_value, set_dbcol_name)
Meta classes? class decorators¶
The latter are much simpler
The latter can do almost everything the former can
- only a metaclass can add methods to the class itself
class-level services (methods)
- @classmethods provide them to instance
- metaclass methods provide them to the class itself
only a metaclass can add to class attrs not visible to self
- meta-methods
- meta-properties
Class decorated can be (more easily stacked)
MISSED BULLET
Example using class decorator¶
def CallTracer(attr):
""" Do custom logic stuff here """
return attr
def tracecalls(cls):
def my__getattribute(self, name):
attr - super(cls, self).__getattribute__(name)
return attr if not callable(attr) else CallTracer(attr)
cls.__getatttribute__ = my__getattribute
return cls
@tracecalls
class MyClass(object):
pass
Diving into attribute manipulators¶
The talk dived a bit into things like:
- __getattribute__
- __getattr__
Diving into this sort of code is tricky, because the reasons for use of these tools is not necessary in 99% of Python projects. I prefer to rely on decorators to alter behavior because they are syntactical sugar. Easy to find and very explicit.
relate or !relate¶
by Mark Ramm
- “Python has been very good to me”
- Double major in Philosophy, Literature and minors in Theatre and something else
- Wrote the TurboGears 1 book and became a web developer
Opening¶
Tools matter
- Python is a great tool
- “SQLAlchemy is the best ORM on the planet regardless of language”
Data matters just as much
Tools matter¶
Know your tools
Screws and nails
- deck (screws are stronger than nails, but harder to use)
- siding (nailing yourself to the siding)
- mongodb (nailed the SourceForge team to the wall via MongoDB)
Takeaway: If you don’t know your tools when you hit production you are going to be trying to debug something critical.
SQL¶
“Who here has used a non-relational database? Not many? Oh, there’s this thing called a FILESYSTEM.”
ACID¶
- Atomicity
- Consistency
- Independent
- Durable
Why I NEED relational¶
I can’t use NoSQL because
- It’s financial data (need consistency)
- my data is relational
Mark says BULLSHIT
- Amazon tracks financial data using NoSQL
- You can make things more durable and consistent if you know what you are doing
Reason to use SQL¶
- Well known, easier to find people who know it
- Robust, scalable, flexible, simple
- pretty ACID
Really good reasons to use RDBS¶
DSL for ad-hock queries understood by everyone
- Business and marketing types can run their own queries
blah blah inconsistencies blah blah latency
Being down is better than being wrong
- Critical for medical applications!
NoSQL¶
Many different types:
key/value store
distributed key/value store
column oriented stores
map-reduce store/system
documented oriented store
- MongoDB
- CouchDB
- XML (now we use JSON instead of this)
- ZODB (A really good system in Python - just so long as you don’t want to share your data with other languages)
graph oriented stores
CAP theorem¶
It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:
- Consistency
- Availability
- Partition tolerance (the system continues to operate despite arbitrary message loss)
Google and Facebook scale has to handle downtown gracefully so has to choose which of these three things they’ll focus on
Focused¶
NoSQL systems tend to have a few but not all of the following bullets:
- Scalable
- Simple
- Fast
- Flexible
- Topic
It’s WebScale¶
Mark calls BULLSHIT on WebScale
99% of sites don’t care about this issue
99% of people are okay if their sites have a feature that fails on a continent
Think about what you actually need
- Don’t implement a database just cause it’s cool
merciless.sourceforge.net
SQL version:
select * from document where x=3 and y="foo"
MongoDB version:
b.things.find({x:3, y: "foo"});
Conclusion¶
Figure out what YOUR app needs
- SQL, MongoDB and Cassandra is great at some things
- Mark couldn’t answer this for Hadoop
Don’t obsess about SCALE you’ll never achieve
Launch your project, then worry about scale