War of Attrition – 2014

Late 2013 – mid 2014 was a busy busy time. I was employed full-time, but was not happy with pay or benefits. I interviewed at several high-powered places, and a startup. It was a lot to keep up with. I made this table to keep track of everything.

Date Apple Klink Mozilla Google Amazon Golden Frog New York Yankees
2013-12-02 Interviewed in Cupertino Friend says I should apply
2013-12-04 Applied
2013-12-05 Keep me happy email received
2013-12-09 Another keep me happy email
2013-12-11 Friend refers me to VP who is another friend
2013-12-13 Told me that they would make permanent offer
2013-12-20 Sent email to hiring manager for several positions
2014-01-03 Accepted permanent offer Contacted me via LinkedIn
2014-01-06 Turned down Phone screen
2014-01-08 Hiring manager sends me email
2014-01-09 Applied and immediately turned down
2014-01-10 Talked to manager of one team
2014-01-13 Email received
2014-01-14 Talked to friend from Google
2014-01-21 Contact from LinkedIn
2014-01-23 Another phone screen
2014-01-27 Technical Phone Screen Phone Interview
2014-01-28 Technical Phone Screen
2014-01-29 Technical Phone Screen Technical Phone Screen Turned down
2014-02-10 Onsite Interview – Seattle
2014-02-18 Onsite Interview – Mountain View
2014-02-20 Onsite Interview – Mountain View
2014-02-21 Turned down
2014-03-03 Offer made Turned down
2014-03-07 Offer accepted
2014-03-10 Announcement that Klink was shutting down
2014-04-08 Mozilla CEO scandal – manager reconfirming acceptance
2014-04-10 Last day at Klink
2014-04-21 First day at Mozilla

So, at the end of this process, the company I had been working at disappeared, and I got only one offer rom the other places I interviewed at. The real bummer is that all of the other opportunities were software developer or software manager positions, but the offer was for a QA job. That was disappointing. Nonetheless, I needed the work.

Welcome to Mozilla.

Google – 2014

Right after the new year, I got a message on LinkedIn:

How are you? I see you are at Klink, is that still correct? We know there is awesome talent coming from that company so naturally I wanted to reach out to you.

I’d love to have a chat with you about some of our confidential projects at our headquarters in Mountain View, California and ultimately see if your interests align with things we have going on over here at Google right now, or in the future. I’m available most days, what time and day would work best for you?

Look forward to hearing from you!


So, the last time I had interviewed with them was in 2000, which was before they went public, and before they became “Google” and started interviewing “the Google Way”. I also applied to them in 2008 when they had their first Austin office, but they turned me down. In the 14 years since I first interviewed there, they had become one of the most famous companies for software engineers. The thing that Google did from the very beginning is: They got rid of schedules.

There is a lot of literature out there about scheduling and managing software projects, starting with https://en.wikipedia.org/wiki/The_Mythical_Man-Month written about the OS 360 project at IBM in the late 1960’s. The conclusion that all of them have reached is that scheduling software is hard/impossible. There is a lot of literature that hard dates are one of the most destabilizing constraints on a software project. So Google got rid of them. They embraced software management methodologies such as Agile are a way to promise small, discrete deliverables. They had started writing books and giving talks about how they did things.

In 2000, they were just another startup, with all of their engineers on a death match towards an IPO. In 2014, they sounded like a software paradise.

And they were recruiting me. Wow!

I was star-struck. I knew that if I was hired by them, I would have to work in California, but from what I understood, it would have been worth it.

I replied that I was interested, and the recruiter wrote back:

Thank you for getting back to me so quickly and I am glad to hear you think so highly of Google and that you are interested in possibly exploring opportunities. I am looking forward to chatting with you soon.

How does your schedule look next week? Does Monday, January 6th work for you at 9:30am PST? Let me know if that works for you.

An updated copy of your resume along with the names of any current Google employees you know will definitely help us to streamline our conversation and the process as a whole.

Looking forward to chatting with you soon and happy new year!

I sent her a list of 19 people I knew were currently working there, and my updated resume. And we talked. She talked about their organic product model. She also mentioned that if they did make an offer, I would have 90(!) days to report. The next step was a technical phone screen.

And they sent me an email with all kinds of information in it, such as the following links:

And then, they sent me a tome on interview preparation. The first part:

Interview Prep:

First off, don’t be intimidated.  A lot of candidates find the interview process to be challenging, yet fun! That being said, candidates who spend ample time preparing using the below guidelines tend to do far better than those who do not.

There were then pointers on planning ahead, what to expect, how to succeed, what Google is looking for, further reading, and technical preparation tips, including studying complexity, sorting, hashtables, graphs, math, operating systems, coding. Coding could include constructing/traversing data structures, implementing system routines, distilling large data sets into single values, data transformations, …

And it finished with:

I know this was a lot to read, but I wanted you to be prepared.

It was very long email with a lot of detail, and despite the admonishment not be intimidated, it was totally intimidating. So I started reading my old algorithms book, but realized that a book where the pseudo-code was written with Pascal might be a hindrance. I bought the more modern version in Java, Algorithms, 4th Ed (Sedgewick/Wayne)

Then, one of my old friends from Apple who was working at Google got in touch with me, and chatted with me about the interview process at Google. It was good to hear from him, but he basically repeated what that long email had said.

So I did the phone screen. A man from Singapore called me(!). The problem was a reasonably simple graph algorithm problem. I thought I did well on it.

They took quite a while to decide on next steps. I told them truthfully that I had two onsite interviews scheduled in the next couple of weeks (one at Amazon, and one at Mozilla). She acknowledged that, and said that she would “try and speed up our process on my end as much as possible.”

A couple of days later, I was informed that they wanted to interview me onsite! That was exciting.

The logistics were fun; I was already interviewing at Mozilla in Mountain View on the Tuesday after President’s Day. Working with both companies, we agreed that they would split my travel costs, and that I would interview at Google Thursday after President’s Day.

I got yet another copy of the long email about preparing for the Google interview, except they told to expect 5 45-minute technical interviews with a Google Engineer.

The day finally came. I arrived at the GooglePlex to discover that it was really different than when I went in 2000.

Googleplex Google Headquarters Aerial
The Google campus reminds me of the home screen on Android. There is all kinds of stuff their, but it is kind of just thrown together. Apple’s campuses are like the iPhone home screen. Tight, neat, and all of the pieces are designed to work together.

Looking back on it now, I can see that I did not do particularly well.

Session 1 was some kind of string processing problem. I remember doing well on this one.

Session 2 was about what it he minimum number of squares/moves that a bishop would need to move on a chessboard. I tried to solve it with math; probably would have done better using a graph.

Session 3 was an architecture session. I never know how to gauge these; I still don’t really have enough experience in this area to do super well on these.

Had lunch with an engineer. They were totally boring. Gave one or two word answers to all of my questions.

Session 4 was all about figuring out how many regions a given point touched. I had a brain fart on the exact terminology for exponential complexity, but this seemed fine.

Session 5 was a disaster. The interviewer’s English was really hard to understand. The problem seemed really really basic, but required a large amount of code. I kept trying to find clever ways to avoid writing all of that code, but the interviewer was having none of it. I wrote and wrote and wrote, and I still feel I missed some point somewhere that would have made the problem easier…

And that was that.

The next post will wrap up the late-2013/early 2014 job search, with a timeline, and what actually ended up happening.


Mozilla – 2014

Neat. An open source company. I had been missing that.


After the day of interviewing at Apple, I had dinner with a good friend who I first worked with at Cygnus. After discussing the individual interviews, he asked if I were interested in working at Mozilla, the non-profit responsible for Firefox. He and I had worked on open source software at Cygnus, and Red Hat, so I was intrigued. He mentioned that his former boss at Cygnus, a truly great manager, was the head of QA, and that we had another couple of friends who worked there as well. I told him I would look at job listings when I got home.

I applied to five positions, and used his name as a referral. However, Mozilla told me that nobody was interested in any of the five positions I applied for. My buddy then told me that I should contact our mutual manager friend.

I managed to get that guy on the phone. We caught up a little bit, and then I told him that I had had dinner with our friend, and that he had expressed interested in my applying to Mozilla. VP of QA friend told me I should email this other manager, who, in the interest of not getting lost in all of the different people I talked to, I will call Cliff. I sent Cliff an email right before the winter holidays, and waited to hear.

Right after the holiday, Cliff put me in touch with a hiring manager on the Firefox OS team. Firefox OS was Mozilla’s attempt to write a phone operating system, whose apps were all based on HTML5/CSS. They were ramping up automation testing. I talked with that manager, and things seemed to go well.

The next person in line, a few days later, was a veteran QA engineer. We talked about testing methodologies, and the like.

I then talked to an engineer that I had worked with at Cygnus/Red Hat. We talked about strategies for automating testing. He described to me a rig he built to test the camera phone made out of Lego Mindstorms. That was fun.

Phone screen #4 was another veteran QA engineer. He asked me the following:

“You have 2 bugs that you think should prevent shipment. You are about to go into the final release meeting. How do you break the news?”

I basically answered that it was a situation which should have never happened. The only time this should happen is if the bugs were found less than 5 minutes before the meeting. If there are bugs in the product that serious, they should already have been discussed by management, and a decision should have already been made about them. Bringing it up in a meeting like this does nothing but alienate everybody from you. It might get the bugs fixed, but it might also get you fired. I don’t want QA to be a Product Prevention department; we are all trying to ship good software here.

We then scheduled an interview session in Mountain View at Mozilla headquarters. I had six interviews scheduled that day. My old friend who played with cameras grilled me about Python. The guy who asked me the above question asked me to find problems in some Javascript code. I don’t remember much about the other QA interviews. The last interview was with a Director of Engineering, a person who had joined Mozilla when it first spun out of Netscape and had worked his way up.

We had a good time. He asked me to do a programming whiteboard. I came up with the naive solution in about 10 minutes. I spent a long time making several performance improvements, analyzing complexity as I went. We discussed hash tables and GUIDs. We discussed C++ vs Java, and everything vs. Javascript.

This interview was scheduled for 45 minutes, but he talked to me for at least one hour and fifteen minutes. At the end he said,

I have one last question. Dev, or QA?

I had to pause a minute. Then I answered:

I am interviewing for this QA job. I know it’s there, and if I get an offer, I need the job. Is there a dev opening? Would it prolong the interview process?

He said:

Yes, it would.

I continued.

Assuming I did a good job for a couple of years, how hard would it be to transfer to a software engineering roles?

He responded:

That happens all of the time.

And so I told him that I was going to continue to apply to this QA role.


I was interviewing for several jobs at once during this time. I am going to wrap up the results of these in one post, after I discuss them all.


Amazon – 2014

Oh, boy. So excited.


I got a message on LinkedIn:

Hope things are going well with you these days. You have a very interesting background stemming from your degree at Rice University. I just wanted to see where you are at with your current engagement at Klink and if you have any interest in any of the opportunities I represent here at Amazon in Seattle, WA. If you do maybe give me a brief idea of what your ideal position looks like and I can let you know what roles we have that would be in line with that. If you are currently happy and not interested in hearing more no worries just wanted to touch base anyways.

I responded with interest, and sent my email.

After some back and forth, he sent me a job description that he had picked out for me.

Job Description

Amazon Identity Services is looking for software engineers who like to solve complex problems, and relish the challenges of building and operating complex, distributed, mission critical systems under extreme loads. Our systems manage hundreds of millions of records, and serve millions of service requests. Do you think you are up to the challenge?

Identity Services provide the core services that identify, authenticate and authorize our customers, and provide the information to hundreds of services within the Amazon service oriented architecture. We manage the customer authentication and authorization experience, and are embarking on new and exciting initiatives in this space, both on the web and on mobile devices. If you are excited about solving new business problems using state-of-the-art technologies, and open standards such as OpenId and Oauth, or developing applications and frameworks for mobile platforms, we’d love to talk to you.

Duties will include design, implementation, documentation, support and operations of various facets of Identity Services. We are looking for self-starters who are interested in leading new initiatives. Successful candidates must also be innovative, creative, flexible, self-directed, and able to design and write high-performance, robust and maintainable code. They must have excellent verbal and written communication skills. The ability function at a very high level in a fast paced environment along with a team of very talented engineers is essential.

If you enjoy working in a dynamic environment to deliver world class mission critical systems, this may be the career opportunity for you.

It goes on to say things like they want somebody with a CS degree, and programming experience, and close with “Ability to handle multiple completing priorities in a fast-paced environment.

I can do that. Then came the phone screen. It was a live programming session in a shared document system.

The person on the other end game me the basic problem: You have a list of star coordinates, and you need to find and print out the closest 50 stars to Earth. I asked if I could have a PriorityQueue API, and then was able to complete the exercise with time left over. That was fun.

So, a few days later, I got an email that said, “The hiring team has really enjoyed speaking with you and we sould like to schedule a time for you to come to Amazon for in-person interviews!…” (Yes, they did not put a comma in that sentence.) It went on to describe roles which were really senior, and which I did not actually feel qualified for. The recruiter assured me that not many people outside of Amazon’s core engineering team was qualified when they started, so please come in.

So I flew to Seattle.

It was February, and it had actually snowed the night before. Doesn’t normally snow much there, so there were many traffic problems. The interview was in a tall building downtown. The building had a really great view of the Space Needle, so it had that going for it. I went to the lobby on whatever floor it was. There was a holding area just for interviewees. You had to show ID, and then they printed a visitor’s badge. The entrance to the interview rooms was strictly behind a badge lock.

They took me in. They told me that if I needed to use the bathroom, somebody would have to walk me there and back. Lovely. Felt like a prison.

I had three whiteboard programming problems, and one architecture problem. I would be surprised if my answers to the architecture problem were worth anything; I felt lost.

I don’t remember one of the other problems. However, one of them was a computational geometry problem. I spent 20-25 minutes trying to figure out the definition of a “ray”. It did not help that English was not the interviewer’s first language, and he had a really hard-to-understand accent. That was frustrating.

The other one was this: Given an arbitrary object, write something which would output the object in JSON format. This is not hard, but it is very, very long. I wrote and wrote and wrote and filled three white boards and really did not see the point at all.

That day did not go well.

I had several onsite interviews all bunched together. Resolution of this interview will be in a future post.


Golden Frog – Early 2014

This was my favorite toy as a kid!

When Klink hired me as a contractor, it was supposed to be for 3 months. As the holidays for 2013 were approaching, that three months ran out.

I had a meeting with my boss. He told me that they hired me because the original Mac developer went AWOL one day, and the person that they had wanted to work on the Mac went on family leave, and would be gone three months. She was coming back to work, so they had originally wanted my contract to terminate after 3 months.

(Shudder. I knew that the contract was temporary, but it is still chilling to hear somebody talk about you as a plugable resource.)

However, I had impressed them so much as a Mac developer, that they were going to keep her where she had good skills (backend work), and would like to make a permanent offer for me.

He asked me what I wanted in salary, and I told him, and also said that I wanted Klink to buy me a work machine, since I was using my personal laptop at this point.

Right before the holidays, he presented me a written offer for $5000/year less than the number I had told him. There was no secondary compensation at all. No bonuses; no equity. There was a health plan, and a 401K plan, but still, the offer was pretty disappointing. Not having a job in my back pocket, I accepted, but started working on another position immediately.

As 2014 unfolded, the new Mac for me to work with never developed. And then we lost our lease on our office, because our shadowy corporate masters did not want to sign a 3-5 year lease on a new place. The office moved to a shared office co-op. I asked if I could work at home, since I had worked at home a lot before, and it would mean less space in the shared office. I agreed to be available on Skype, and come in once a week or so for meetings.

So the first part of 2014 was a whirlwind. I am not going to write about chronologically, since everything was happening at the same time. It was hard to keep up with.

One day, I got a phone call for a company looking for a Mac developer. Golden Frog was a local company, and they had a file syncing client to their cloud storage system, much like Klink (or Dropbox or Amazon Drive or Box or Google Drive…) I downloaded the app and took a look.

The app icon was a cartoon version of a Tonka dump truck. It offered what you would expect, but had no “Klink this!” feature, which would establish a link to an arbitrary folder to your sync set. Still, seemed like what I was currently working on. Digging around in their app bundle, I figured out that they were using the same python library to keep track of file system changes.

The recruiter set me up with a phone call with the manager. She described to me the organization and business plan. There were 20 people in the engineering department, all reporting to her. A small number were working on DumpTruck, but the others were working on their proprietary VPN product. In the conversation, she mentioned that the senior engineers were not listening to the product team/upper management, and that she had no visibility to what the QA and release engineering teams actually did.

She also informed me that I was too senior; the budget was for a junior engineer.

I then questioned her:

Me: How many direct reports do you have?

Her: 20

Me: Are there any other managers?

Her: No

Me: What kind of product scheduling methodology do you use?

Her: We are trying to move to Agile, but have some rought patches.

Me: When was the last time anybody took a significant vacation

Her: I don’t know.

I knew I did not want to be an engineer here anyway.

Me: So let me guess as to what is happening here. The engineers are confrontational because they don’t think that they have time to do real work, and there To-Do lists are way too long. QA is complaining because they already have too much to do, but they don’t know what to test with anything new. They are also pushing for automation, but have nobody who can do that work. Release Engineering is pushing for a continuous-delivery system, but since there is no automation, and probably no budget, it is falling on deaf ears. And your management is upset because the team appears to not be delivering anything.

(10 second pause)

Her: Go on.

Me: I am going to boldly assert that I can help you, but as a manager. I have experience managing both engineering and QA teams, and have managed full-cycle development. I have done performance evaluation, product design, and systems design. If you were to hire me, we could work together to make this team much better.

Her: Your guesses as to what is going on around here are scary.


Her: We don’t currently have any budget for another manager, but let me get with my management and talk to them. I’ll get in touch later.

I scared her. I never heard from her again.


New York Yankees – 2014

I mentioned in Project Scoresheet – 1989 that I was an amateur baseball statistical analysis (“stathead”), and dreamed of working in an enlightened front office someday. At time of that post, 1989, there were no front offices truly embracing analytics. Moneyball documented how one club started the process, although Oakland wasn’t alone. By the time the mid 2010’s came along, most clubs were starting to hire data analysts and programmers.

I still get a cold chill when I see this logo, though.

I saw a job listing on some board somewhere for a job as a programmer for some team or another, and when I cliked on the link, it took me to Teamwork Online, which is a site that most of the major league clubs use to post jobs. I signed up for that site, and notifications for analysis and programming jobs.

Over the years, the small handful of positions that were sent to me were not all of that interesting, but one did stand out. The New York Yankees were hiring a mobile developer to work on its internal apps. What the hell? I applied.

A few weeks later, I received the following:

We want to thank you for your interest in the above mentioned position. We had many fine applicants for the position, including you. However, we have decided not to fill this position. We welcome you to apply for any future positions we have available that match your skills and experience.


The Hiring Manager for the “Mobile Application Developer, Baseball Operations – New York Yankees (Bronx, NY)”
MLB Baseball Jobs

Moral of this story: A significant number of job postings disappear before being filled. I do not know if I would enjoyed living in a land which had real winter, but it was slightly disappointing not to be interviewed.

Apple – 2013

Klink was fun. Programming on the Mac was fun. Learning Python was fun. Dealing with the Python-Objective C bridge was not so fun, but you can’t have everything.

However, this was still a contract, and although they had talked about converting to permanent at the end of the contract, I had nothing close to a guarantee. So I had to keep my eye out for another position.

A couple of recruiters contacted me about IOS positions, but they required me to move, or the company looked awful, or any number of other things made those contacts not desirable.

After a few weeks, I got an email from a recruiter at Apple:

I am a recruiter with Apple and recently received a copy of your resume for review.  Would you have interest in speaking with an Engineering Mangers of ours, Dave Chan regarding an opportunity we have for an iOS Performance QA Stress Test Manager? Unfortunately I do not have a job description for you, but Dave can walk you through during your call the role and expectations.  What might work for you this week for a 30 min call?
I look forward to hearing from you.

There were problems with this, of course. It was in Cupertino. It was leading a team of QA and automation people.

I really did not feel like I had a choice but do to pursue this and see where it lead. So, I talked to the hiring manager. It was a non-technical question-and-answer session about my career to date. He suggested I talk to another person, the head of performance engineering. We talked, and had a lively discussion about what performance testing means.

So they decided to fly me out for a full round of interviews. Got an Apple looking email stating “We would like to confirm your interview for the position of IOS Performance QA Stress Test Engineer”. I guess somewhere along the way they decided not to pursue me as manager. Whatever.

I flew out on Sunday for a Monday interview, with my return flight being a red-eye on Monday night.

The interview seemed pretty ordinary, in that, at the time, they only did one technical interview. They had me solve some kind of problem in Python, and I did not do particularly well, considering I had 3 months of experience in Python. Each interview had 2 people, and while that seems normal to me now, it was jarring then.

It was weird being back on the Infinite Loop campus. Parking was almost non-existant. The cafe was very very crowded. One of the strange little things: All of the bathrooms now had the metal wedges bolted to the wall, about 1 meter off of the ground. They were about half a meter long, and they flared out to 20-30 cm on top. People put their laptops, iPads, and notebooks in them while they did their business, and retrieved them after they washed their hands. Wish more corporate bathrooms had them!

I interviewed at the beginning of December, and they kept putting off the decision and putting off the decision, and then they told me I had to wait until after the holidays. Finally, I got this:

Thank you for all of your patience through our process.   I was able to debrief with [Hiring Manager] and unfortunately the team have made the tough decision not to move forward.    While they feel you have some great experience, our role is not the right match for your current background.  I am sorry this role will not work out for you.   Are you currently speaking with any other teams here at Apple?   If not I am more than happy to share your resume to broader recruiting team for review.
I am sorry to not have better news for you and I do wish you all the best.

So. No Apple.