Everything I know about Python...

Learn to Write Pythonic Code!

Check out the book Writing Idiomatic Python!

Discuss Posts With Other Readers at discourse.jeffknupp.com!

Sandman, Rebooted: Create a REST API For Legacy Databases Without Writing Code

sandman is by far my most popular project. I think there's a good reason for that: it does something genuinely useful. Ever had to deal with a legacy database through some awful system or API? sandman auto-magically creates a fully RESTful API service for your legacy database without you needing to write a line of code. You simply enter the connection details for your database at the command line, hit enter, and "hello REST API!".

That's all great, but what about the code that powers it? I'll be the first to admit that sandman, while well-tested and well-documented, is implemented in a bit of a crazy way. The reason for this is a combination of my limited knowledge (at the time) of SQLAlchemy and a few shortcomings in SQLAlchemy's reflection capabilities. I had to do some code gymnastics to get it all working properly.

But SQLAlchemy released a new major version (0.9) with a new, if under-publicized, feature called "automapping". This is as perfect a fit for sandman as it sounds: you can use it to automatically create classes based on reflected database tables. Once I saw this, I instantly realized this could be a boon to simplifying sandman. Still, though, I hadn't been writing REST APIs for very long and didn't have a great handle on how to generalize things enough to properly make use of automap.

That changed in the past few months, though, and in the back of my mind I began to hear the siren's song calling me to rewrite sandman from scratch. The problem, though, was the original sandman worked. It worked due to awesome people entering Issues on GitHub and me feeling bad that my baby had problems (my sandman baby. Not Alex, my real baby, who is perfect). I didn't want to lose all the work I had put into sandman.

Finally, yesterday, I decided to pull the trigger and see just how small I could make sandman. As it seems to be the most used feature, I decided to focus just on the case where the user wants sandman to reflect all of their database tables and doesn't want to customize anything. How quickly, and in how many lines of code, could it be done?

Read on →

Omega: The Last Python Web Framework

The preparation for my talk on Monday at the Wharton Web Conference got me thinking a lot about the impedance mismatch between web frameworks and REST APIs. The more you develop REST APIs with frameworks like Django or Flask, the more clear it is that something is amiss. It's a subtle feeling of needing to fight against the grain to accomplish your goal.

Which of course led me down another rabbit hole: server-side web frameworks in general. There are two things they pretty much suck at right now: helping you write REST APIs and supporting bidirectional communication over things like WebSockets.

Read on →

Your Database's REST API: Sandman One Year Later

A year ago yesterday, I made my first commit to what would eventually become sandman ( official homepage ). It remains my most popular Python project by a wide margin. Why? Because it solves a real problem that I've faced many times. July 14, 2013 was just the day I decided to do something about it. And so, as I sit in my hotel at the University of Pennsylvania writing this post, preparing to give a 90 minute talk on Python, REST APIs, and sandman, I'm struck by how far the project has come in a year and the effect it has had on me.

Read on →

Exciting Progress Made on 'Python's Hardest Problem

A long awaited announcement was finally made Saturday by the PyPy team: PyPy-STM has reached a reasonable level of maturity and usefulness to begin comparison to PyPy and cPython. We can consider the first (C-based) GIL-less Python (disregarding that old "no GIL" patch from years ago which went nowhere).

The implications are important: Armin and team have proven that Software Transactional Memory is a viable approach to GIL-less multi-threaded Python. They are now ready to move to the second phase of the project, namely improving things to the point that there are no caveats to the statement "PyPy-TM can run existing multi-threaded Python code without a GIL and achieve reasonable speed-ups".

As someone who has followed GIL-related work for years (albeit from the sidelines) I can't understate how excited I am. Though Python continues to enjoy a reasonably high rate of adoption (I assume) across a number of disciplines, the GIL always made taking full advantage of today's multi-core hardware difficult at best. In a GIL-only world, Python may be discarded for similar languages that do not share the same drawback, especially as they mature.

Today's announcement, however, has made clear a road ahead for Python. If Armin and company are able to complete the second phase of the project (scheduled to last up to two years), I can imagine a world where PyPy becomes the reference implementation of Python. As crazy a statement as that sounds, that's how important I believe solving this problem is.

In general terms, today's announcement was a win for developer ingenuity and perseverance. Work on the STM branch began years ago, and there were a number of false starts. The team kept at it, though, and showed the rest of us that the hard problems are worth working on, assuming enough dedication (and not a little bit of intelligence and skill!). It's a win for those operating on the fringes of their fields, on the problem that few discuss and less try to solve. Most importantly, it's a win for the Python community at large.

Good luck to Armin and Remi (who are both committed to phase two assuming there is funding), as well as all who join them and contribute development time. I, for one, will be donating to support their worthy research.

Read on →

Web Analytics