Blog

Roy Fielding’s advice on versioning APIs is, well, succinct:

Don’t.

In a subsequent interview, he elaborates:

Websites don’t come with version numbers attached because they never need to.

Neither should a RESTful API.

Analytics and machine learning are increasingly relevant tools in the software professional’s toolbox. Statistics and probability now play a greater role than ever in our work.

Games can be a fun “gateway drug” for learning these techniques. In particular, the sports industry has recently turned to mathematics in the quest to gain a competitive advantage. Organizations like FiveThirtyEight, originally known for using polling data to make predictions about political races, now routinely predict outcomes for sporting events based on performance models and historical data.

Recent buzz about Facebook React—I guess there was a conference or an announcement or something?—inspired an internal discussion here at Panda Strike. We’re on the record as not being big fans of the framework, but, as a group of smart, spirited software professionals, we sometimes disagree. Our respective opinions on Facebook React, and possibly many other things, may differ from those expressed on our blog. Which is fantastic. In fact, we’re hoping to bring more of those voices to the blog.

HTTP is the world’s most successful application protocol. Yet it is widely maligned and misunderstood. Part of the problem is a poor developer experience. Part of the problem is that a Ph.D. dissertation is usually not the best introduction to a subject. And part of the problem is that building network applications is hard and we blame HTTP for that when we shouldn’t.

The HTTP protocol gets a bad rap. HTTP is simply an application protocol that provides a well-considered set of features for distributed applications. Naming, caching, compression, and so forth, are not superfluous complications introduced by HTTP. They’re just things distributed applications need to do. HTTP is annoying because distributed computing is annoying.

Troy Hunt recently wrote a blog post with the wonderfully descriptive title, It’s 2016 already, how are websites still screwing up these user experiences?!. Many of the problems Mr.Hunt describes are manifestations of content producers inability to figure out how to make money.

On the one hand, if you’re selling ads, you can end up crossing this weird line where you’re basically telling your readers that you don’t really want them to read your articles. You simply want them to click on an ad. Your content has become subordinate to the ads. Is that really what you wanted?

Estimating the cost and duration of software projects is, apparently, a hard problem. Thus, Hofstadter’s Law:

It always takes longer than you expect, even when you take into account Hofstadter’s Law.

I was reminded of this, and inspired to write this blog post, when I saw an Web app pop up in my Twitter feed. It’s called Guesstimate and it describes itself as “a spreadsheet for things that are uncertain.” There’s even a blog post explaining the idea. It’s a fantastic way to explore the impact even small amounts of uncertainty can have on costs.

Developers are struggling to fully leverage HTTP and REST. By definition, we can stipulate that they’re difficult technologies. This is largely because distributed computing is hard. And HTTP and REST are distributed computing technologies. If we want to solve this problem, we need make things easier. In particular, we need better tooling and libraries that make it easier to leverage HTTP and REST by default.

Our clients hire us because we create value for them. But that value isn’t always obvious to decision-makers and stakeholders. They may believe finishing a given project is possible with fewer resources or in less time.

Prior to becoming a Panda, I studied as a theoretical chemist. So, while my computer science education is not exactly traditional, I learned a great deal about applying computers to physical models.

The toy I’ve built for your entertainment honors both the old and new in my life. The browser conducts a 2-dimensional simulation of free particles that exert an attractive force on each other. The resulting dynamics are rendered, live, to the screen via an HTML canvas element. Image courtesy of NASA/JPL/Caltech

Back in 2012, I wrote a blog post introducing the idea of Web capabilities. Once in awhile, someone asks me what happened to it. The answer: I don’t know. I can’t even find it on archive.org.

Since then, the rise and fall of Bitcoin introduced the capability security model to the world…

Apple recently integrated ad-blocking capabilities into iOS9. This triggered a spate of articles predicting the end of the world as we know it. The only question was whether the ending was a comedy or a tragedy.

I’m here to tell you: this ad-blocking business is the best thing ever. Empowering consumers to limit ad inventory will force publishers to declare their true colors. It will blunt the incentive to flood the Web with click-bait. Ad rates will rise as inventory drops. And legitimate publishers will be able focus on providing compelling content.

A long-running theme on this blog is that the Open Web matters. And, further, that the Web isn’t broken, it’s just fine, thank you very much. This includes JavaScript, which is, of course, a central part of the Open Web stack. Although it might be more fun to laugh about peculiar edge cases, or bemoan the size of the most recent spec, the reality is that JavaScript is a powerful and widely deployed language. And, ladies and gentlemen of the jury, we will show not only that JavaScript is not broken, but, further, that the recent evolution of the language is nothing short of genius.

Our recent post about Relay and REST raised some interesting questions.

Bickering about REST aside, doesn’t Relay solve some real problems?

Yes. But Facebook set up REST as a strawman. There’s enough confusion about REST already without adding to it.

But aren’t data-oriented APIs like Relay and GraphQL at odds with REST?

We don’t believe so.

Okay, but what about cache invalidation? You don’t address that.

Web Components are awesome, because component-oriented architectures are a proven model for interaction design. (For example, the two major mobile operating systems are arguably component-oriented.) Also because they are being defined as part of an open standard, and, as such, will enjoy ubiquitous support from browser vendors. In short, they’re a long overdue part of the Open Web stack. Unfortunately, the “Web is broken” angst plagues Web Components, just as it does every other part of the Open Web. We’re here to help.

Data-oriented API frameworks, like Relay and Falcor, are trying to solve problems for which HTTP APIs appear, even to experienced developers and architects, to be poorly suited. Let’s put aside spurious claims about HTTP and REST, and explore these concerns.

The most obvious difficulty is dealing with ad hoc queries. Semantically, a query ought to be a GET request. But HTTP caching relies first and foremost on the URL for caching. And mapping the semantics of a sufficiently rich query language into a URL seems both contrary to the notion of opaque URLs (and thus REST) and just plain awkward. We can, of course, use POST and send the query in the body of a request, but we lose the benefits of HTTP caching in the process. (You can, in certain circumstances, cache POST responses, but that won’t help us here.) At first glance, then, if we want ad hoc queries with cacheable responses it appears we cannot rely on HTTP.

Let’s try a little harder, though.

I’ve never had any interest in being a REST or HTTP evangelist. However, I believe that the Web, and the Internet upon which it stands, are possibly the most significant technological developments since the printing press. That’s why the Web-is-broken meme bothers me so much. The Web isn’t broken—it’s better than ever. It isn’t perfect, but I don’t believe that the answer is to go back to the world of proprietary islands of technology. However, the Web’s connection to the REST architectural style wasn’t always obvious to me. In fact, until recently, I couldn’t have explained it to you.

Facebook’s on an evil mission. It wants to convince you that the web is broken. So Facebook recently introduced Relay. To explain why you “need” Relay instead of REST, they made a series of claims about why REST is broken.

These claims are false.

A few months ago, I talked about why I don’t like React. One of the reasons is that I believe Facebook, and frameworks like React, are undermining the Open Web. This may not be intentional. I certainly don’t think it is the intention of the developers on those projects. But I’m less confident in the motivations of the people running the companies behind these frameworks. And, either way, they aren’t good for the Open Web.

In our last blog post, we argued that if you want to keep people in a physical office so that you can assess their productivity, then you probably don’t know how to assess developer productivity. And obviously, if your job includes managing engineers, that’s a useful skill to have.

Remote work’s only possible if you can make sense of how much progress your developers are making, whether they’re in the office or not. Physical presence is a terrible substitute for trust and insight. So the question becomes: how do you know how your team is doing?

Developers have to retool their skill sets on a regular basis if they want to stay relevant. This has been true for a very long time and it’s widely considered part of the job. But today, managers have to retool their skill sets from time to time as well. This is a new development.

I once saw an intelligent comment on Hacker News:

Asynchronous communications are best for development teams. Taking out an exclusive lock on every developer’s full attention for 15 minutes is extremely dumb.

Talent is crucial to running a tech company, and hiring remote workers can be a spectacular advantage. In places like San Francisco, with fierce and constant competition for engineering talent, it can be a shock to say “ok, we’ll try hiring remote,” and suddenly get inundated with amazing resumés.

But managing a distributed team is completely different from managing a co-located team.

The practical meaning of “doing Agile development” has changed over the years. To quote two Agile consultants:

From a programmer-centric, Extreme Programming focus in the early days, to a more inclusive approach in the mid-2000s, to a project management and Scrum focus in more recent years.

On the one hand, it’s very healthy for a software development process to acknowledge that engineers aren’t the only experts involved. But I feel that Extreme Programming was healthier than Scrum, and that the shift towards Scrum signals that project managers regained control of a seismic shift that was throwing them out of work.

There’s been a lot of excitement lately about React. React seems to have bumped Angular from the top of the hippest-framework mountain. This is unfortunate, because both of these frameworks are bad for your application’s health. They’re also bad for the entire software industry. For new applications, please, for the love of all things open, use Web components instead.

React’s design is bad. I could give you a lot of specifics—separation of concerns, coupling views with models, the focus on needless optimizations, the importance of supporting open standards—but I’m going to tell you a story, instead. The story is about a startup with a popular product and lots of venture capital. This startup decided that they wanted to make their Web app extra special. Like many who came before them, they decided that HTML and CSS just weren’t quite good enough for them.

The Agile Manifesto was written in mid-February of 2001, and it made the tech industry a better place. But it was written by people, not gods, and its day is fading. Likewise, the Scrum development methodology helps people build stuff, but it has flaws. So let’s talk about what Panda Strike does instead of Scrum and Agile, and why.

We’ll start with the biggest flaw in the Agile Manifesto.

[Bastion host(s)][bastion host] are a useful and important component of a system management infrastructure. A bastion-host, in this context, is actually more properly, but more obscurely, called a [jump server]. In this post I will simply use the term [bastion host]. It is the most commonly used term for the system’s function: a server, which has undergone security hardening steps, that is the operational and administrative control point for systems and hosts in a datacenter (or AWS Region).

On the last point about AWS Regions: I will touch on some powerful capabilities of bastion hosts, AWS Security Groups, and cross AWS account-access that we use here at Panda Strike.

Here’s what we’ve learned so far:

• In part 1, we said that HTTP views the Internet as a big key-value store.
• In part 2, we established that GET, PUT, and DELETE were the main methods, with POST acting as a fallback for things that don’t fit the key-value store model.
• In part 3, we discussed how to discover and dynamically construct URLs to reduce coupling between client and server.
• In part 4, we explored a flexible mechanism, known as content negotiation, that allows the client to choose their preferred content format.

In this installment, we’re going to look at how HTTP helps us optimize performance. After all, network requests are considerably slower than reading and writing from disk or memory. We want to do everything we can to speed things up. As it turns out, HTTP can help us do that.One of the primary mechanisms HTTP provides for doing this is caching. It’s a measure of how confused things have gotten that most of the search results for http caching concern turning caching off. And, indeed, we see this often at clients, due to the fact that, for historical reasons, browsers are very aggressive about caching HTML, CSS, JavaScript, and image assets. But that shouldn’t discourage you from using HTTP caching for your APIs.

Here’s what we’ve learned so far:

• In part 1, we said that HTTP views the Internet as a big key-value store.
• In part 2, we established that GET, PUT, and DELETE were the main methods, with POST acting as fallback for things that don’t fit the key-value store model.
• In part 3, we discussed URLs, the principle of opacity, use of parameters to dynamically construct URLs, and discovery to obtain an initial set of URLs.

In this installment, we’re going to look at how we go from a resource to a representation, which is actual data we can use.

In part 1, we said that HTTP views the Internet as a giant, distributed key-value store. In part 2, we reviewed the semantics of HTTP’s methods, with GET, PUT, and DELETE acting as the main interface, and POST acting as a fallback for things that don’t fit key-value store abstraction. In this article, we’re going to explore the utility of the ubiquitious URL.

Recall that URLs are the keys in our global key-value store and are therefore obviously pretty crucial. However, like much of the world’s most viral protocol, they’re often misunderstood.

In part 1, we said that HTTP views the world as a distributed key-value store.You can make a pretty strong argument that the HTTP model is actually a distributed hash table. The protocol’s designers likely didn’t think in those terms (the first documented use of the term was in 1986 in reference to Linda, so it’s possible those ideas influenced HTTP somehow), but the simliarities are profound. DHT schemes usually rely on a two tier system for resolving keys. A key is hashed to a secondary server, which can actually resolve the reference. This precisely mirrors what HTTP does with URLs, which include a host component. Effectively, HTTP leverages TCP/IP and DNS to resolve a reference to a specific server. This partly explains why it scales so well. It’s an extremely clever idea that we tend to take entirely for granted today. HTTP lacks recovery capabilities associated with DHTs, but, even there, redirects arguably perform a similar function. The URLs are the keys and the values are resources. Resources, in turn, are actually dictionaries of different representations, or formats, for the resource. For example, a video resource might have different encodings, each of which is a representation that can be accessed if you know its media type. In Part 2, we’re going to begin to explore this idea in more detail, starting with the verb we’ve conspicuously ignored up until now: POST.

The HTTP protocol is generally poorly understood.And I’m not even going to get into REST. If you really want to understand REST, read the paper by the guy who coined the term (and, not incidentally, was the principal author of the HTTP specification). Technically speaking, HTTP is a protocol that conforms to REST constraints. So we can just talk about how to use HTTP effectively and not worry too much about REST. There are a variety of reasons for this, but one of them is that it’s a fairly rich protocol and, consequently, writing clients and servers that take full advantage of its features is non-trivial. Naturally, developers either make use of a subset of those features or use something else entirely, like Web Sockets.The danger here is that you end up implementing your own variation of the same features, such as caching. See Greenspun’s Tenth Law. At Panda Strike, we’ve tried to make it easier to fully utilize HTTP’s features with Patchboard, but that presumes you already understand them. This blog post is intended to help you do that.