What We Do

If you’re looking at one of our job ads, chances are you want to know more about what we do — what does a software engineering job at AetherWorks actually involve?

We’re currently building AetherStore, a distributed data store. AetherStore runs over the workstations and servers in your organization, harnessing their unused capacity to create a shared, virtual file system. To a user, we appear as any other networked file system, without requiring any extra hardware.

It’s a wonderfully engaging project to work on because we see such a diverse range of activities every day. From the low level handling of calls from Windows Explorer[1], to the task of breaking up[2], encrypting[3], and dispersing files across the network[4], I’ve covered most of my Computer Science education in some form while working on AetherStore.

Why I Work Here

My own background is in distributed systems (primarily distributed database systems), so the de-centralized and fault-tolerant design of AetherStore has obvious appeal. I love the challenge of creating these systems because you are so constrained by the fundamental properties of distribution[5], and yet it’s still possible to create systems that scale to many hundreds of thousands of machines[6].

With AetherStore our challenge is in creating a shared file system where every piece of data doesn’t have to reside on every machine. We want to spread user data across machines proportional to their size, but we don’t have a centralized authority to lean on when deciding how to do this. And we don’t even know how many machines should be in the system[7]. It’s brilliantly limiting[8]!

For me, it’s hard to imagine a better job. I love the intellectual challenge of creating a new feature or component, and the satisfaction of being able to craft this into a complete, stable software product. It’s  truly an exciting thing to be a part of.

Photo of Angus' Desk
The working environment isn’t bad either!

So, while it’s not easy competing with so many other great New York companies, I think we’ve got a lot to offer. Consider applying!

Our Interviews

My biggest problem with jobs listings (ours included) is that we specify a set of requirements that invariably turn into clichés, and we don’t explain why we need them or how we test for them. So let’s look at a few, and see why they actually matter more than you might think.

“Software Engineer.”

The job title may seem meaningless, but I love this distinction between software engineers and programmers. We want to know that you craft code to a high standard, and that you understand why ‘it just works’ isn’t enough.

In an interview we’ll ask you to review some (bad) code we’ve written, to gauge your code literacy. We’re looking for someone that has an appreciation for clean code and a quick eye for bugs.

“A solid understanding of object-oriented programming.”

We’re building a complex system and we need to make sure that you’re the type of person that can structure code in a logical and maintainable way.  We’ll ask you to do a short programming assignment to get a feel for your general abilities and experience.

“Fundamental Computer Science Background.”

The work I have described in the previous section is challenging, and it requires that you know the relative efficiency of, say, a linked list and an array, but also that you’re capable of creating your own data structures from time to time. For us, the best indicator of this skill-set is an undergraduate degree in Computer Science. In an interview we’ll ask you an algorithmic question that gives you a chance to demonstrate the breadth of your knowledge.

If you do well enough in these questions then we’ll invite you in for a longer interview, asking you to solve a real problem that we’re actually working on in the office.

To Apply

If the idea of working at AetherWorks appeals to you, I’d urge you to check out our available positions. Alternatively, if you have any questions about this post or our interviews, please feel free to email me (first initial, last name[9]).

 


[1] And I mean every single call. Every time you open a directory, right-click on a file, or save a document, Windows Explorer is providing us with a constant stream of calls asking for information and telling us what to update. Since we’re pretending to be a network mount, we have to handle each of these calls, giving responses either from the local machine or a remote copy. This fascinates me more than it probably should, but it gives you some brief insight into the complexity of the operating systems we use every day without thought.

[2] When you store a file we break it up into chunks, both to make it easier to spread data across the network and to increase de-duplication. There are entire classes of research dedicated to finding ways of doing this efficiently. Content-based chunking, in particular, has some really clever uses for hashing (fingerprinting) algorithms and sliding windows, which dramatically improve de-duplication.

[3] We have to encrypt data at rest and in transit, but this is more challenging than in most systems where you have a central authoritative server. Without this, our encryption architecture represents a trade-off between security and usability.

[4] Deciding where to place data is particularly challenging, since we don’t have a central coordinator that can make this decision. All machines must be in agreement as to where data is placed (so that it can be accessed), but it is expensive to allow them to co-ordinate to make this happen.

[6] Constraints can be catalysts for creativity.

[7] Since we don’t know how many machines are ever in the system, we can’t use distributed consensus protocols such as Paxos. These require that a majority of nodes agree on a decision, but if you don’t know how many nodes exist, you don’t know how many nodes form a majority.

[8] The CAP theorem is my favorite (trivial) example of this. Imagine you have 3 machines, and have a copy of some data on each machine. How do you handle an update to that data?

How we handle this update (and anything else in distributed systems) is determined by our response to network partitions – when one set of machines is unable to contact another. If we use a central lock manager to stop conflicting updates we ensure that the data is consistent, but that it will be unavailable if the lock manager cannot be contacted. If we use a majority consensus protocol, we can update our data in the event of a partition, but only if we are in the partition with a majority of nodes. If we assume that neither of these cases is acceptable, we can do away with consistency altogether, allowing updates to each individual copy even when the others are inaccessible. The fundamental properties of a distributed system limit us in each of these options — it’s up to us to decide which is the most appropriate in any given case.

[9] This is our interview puzzle question!

Growing Pains: Recruiting as a Start-up

New York is pushing hard in tech start-up world right now. Bloomberg’s WeAreMadeInNY campaign received a lot of press attention last week and hopefully this continuing support by the city, alongside the success stories that are published with chiming regularity, will encourage the best grads and experienced pros in our industry to flock here. But it doesn’t appear to be happening just yet. Not for us anyway.

So, who are we — a small-scale start up — competing against for talent?

One idea is that we are pitted against big corporate.  We see a lot of resumes filled with banking development experience. Multiple jobs lasting 18 months to 2 years. Boredom seems to set in and candidates go searching for the next chapter in their career, invariably another bank. The trend goes on…

I get the impression that these candidates are asking themselves the wrong questions. It has been mentioned to me several times that going to a large company aids career progression in a way that can’t be achieved by going to a start-up. Well what kind of career progression? Better salary? Maybe. Better title? Possibly. More responsibility? I don’t think so! I understand different types thrive in different environments, but from a computer science perspective and someone working in software I’d encourage anyone to read James Currier’s open letter to his computer science alma mater at Princeton. And just in case you don’t, below is a short quote that captures the essence of Currier’s message.

“Big companies teach you how to work through layers of bureaucracy and how to solve problems in very risk-averse ways — in short, how to make something happen in their organization. A big company is not the safe career choice. It’s the risky choice. It risks your mind and your life.”

Here at AetherWorks, we have a very talented group who are all involved in the work and decisions that push the company forward. As a result, we all have a collective responsibility to learn, to achieve, and to progress. We are a team. There are opportunities that present themselves here that I believe would represent valuable experience to anyone at any stage of life, let alone new grads. Sit in on design meetings, take part in product planning, generate new sales strategies for our latest products, pick a topic for our weekly research meetings. We have an environment in which anyone can come forward. We all have our areas of specialty, but we also have the opportunity to learn and assist across the breadth of our business. The best bit? If we do something fantastic, then we’re all going to reap the benefits.

Great, right?

We do need people to take the initial leap of faith though. Our latest recruitment drive has been very challenging and it’s made me aware of a few areas we need to work on to change perceptions.

Pile of resumes.
The resumes pile up.

One notable disadvantage we have, versus large corps and more established start-ups, is that we are less embedded in the systems of recruitment that can help attract talent. AetherWorks has had little time to integrate into the NYC community, we do not have a live product just yet (but we will very soon), we have had limited (but very positive) press, we have had little opportunity to create useful internships that may lead to permanent roles, and maybe more than anything, our alumni network connects us to the UK rather than the US.

We continue, however, to strive to address all of these issues. We are using multiple sources to advertise and post our openings and we continue to raise our profile in the city by attending meet ups & conferences, writing our blog and distributing press releases. On top of that, Fay (Director of Business Development) has been sharing her experiences in schools around the northeast, and Gus (Director of Research) is planning on opening up our research sessions to the public once a month (for more info email us).

It’s a long and competitive game, but hopefully we will see some good results soon. Of course if you happen to be searching for a software engineering role, you know what to do!