Mozilla – 2014

Neat. An open source company. I had been missing that.


After the day of interviewing at Apple, I had dinner with a good friend who I first worked with at Cygnus. After discussing the individual interviews, he asked if I were interested in working at Mozilla, the non-profit responsible for Firefox. He and I had worked on open source software at Cygnus, and Red Hat, so I was intrigued. He mentioned that his former boss at Cygnus, a truly great manager, was the head of QA, and that we had another couple of friends who worked there as well. I told him I would look at job listings when I got home.

I applied to five positions, and used his name as a referral. However, Mozilla told me that nobody was interested in any of the five positions I applied for. My buddy then told me that I should contact our mutual manager friend.

I managed to get that guy on the phone. We caught up a little bit, and then I told him that I had had dinner with our friend, and that he had expressed interested in my applying to Mozilla. VP of QA friend told me I should email this other manager, who, in the interest of not getting lost in all of the different people I talked to, I will call Cliff. I sent Cliff an email right before the winter holidays, and waited to hear.

Right after the holiday, Cliff put me in touch with a hiring manager on the Firefox OS team. Firefox OS was Mozilla’s attempt to write a phone operating system, whose apps were all based on HTML5/CSS. They were ramping up automation testing. I talked with that manager, and things seemed to go well.

The next person in line, a few days later, was a veteran QA engineer. We talked about testing methodologies, and the like.

I then talked to an engineer that I had worked with at Cygnus/Red Hat. We talked about strategies for automating testing. He described to me a rig he built to test the camera phone made out of Lego Mindstorms. That was fun.

Phone screen #4 was another veteran QA engineer. He asked me the following:

“You have 2 bugs that you think should prevent shipment. You are about to go into the final release meeting. How do you break the news?”

I basically answered that it was a situation which should have never happened. The only time this should happen is if the bugs were found less than 5 minutes before the meeting. If there are bugs in the product that serious, they should already have been discussed by management, and a decision should have already been made about them. Bringing it up in a meeting like this does nothing but alienate everybody from you. It might get the bugs fixed, but it might also get you fired. I don’t want QA to be a Product Prevention department; we are all trying to ship good software here.

We then scheduled an interview session in Mountain View at Mozilla headquarters. I had six interviews scheduled that day. My old friend who played with cameras grilled me about Python. The guy who asked me the above question asked me to find problems in some Javascript code. I don’t remember much about the other QA interviews. The last interview was with a Director of Engineering, a person who had joined Mozilla when it first spun out of Netscape and had worked his way up.

We had a good time. He asked me to do a programming whiteboard. I came up with the naive solution in about 10 minutes. I spent a long time making several performance improvements, analyzing complexity as I went. We discussed hash tables and GUIDs. We discussed C++ vs Java, and everything vs. Javascript.

This interview was scheduled for 45 minutes, but he talked to me for at least one hour and fifteen minutes. At the end he said,

I have one last question. Dev, or QA?

I had to pause a minute. Then I answered:

I am interviewing for this QA job. I know it’s there, and if I get an offer, I need the job. Is there a dev opening? Would it prolong the interview process?

He said:

Yes, it would.

I continued.

Assuming I did a good job for a couple of years, how hard would it be to transfer to a software engineering roles?

He responded:

That happens all of the time.

And so I told him that I was going to continue to apply to this QA role.


I was interviewing for several jobs at once during this time. I am going to wrap up the results of these in one post, after I discuss them all.


Amazon – 2014

Oh, boy. So excited.


I got a message on LinkedIn:

Hope things are going well with you these days. You have a very interesting background stemming from your degree at Rice University. I just wanted to see where you are at with your current engagement at Klink and if you have any interest in any of the opportunities I represent here at Amazon in Seattle, WA. If you do maybe give me a brief idea of what your ideal position looks like and I can let you know what roles we have that would be in line with that. If you are currently happy and not interested in hearing more no worries just wanted to touch base anyways.

I responded with interest, and sent my email.

After some back and forth, he sent me a job description that he had picked out for me.

Job Description

Amazon Identity Services is looking for software engineers who like to solve complex problems, and relish the challenges of building and operating complex, distributed, mission critical systems under extreme loads. Our systems manage hundreds of millions of records, and serve millions of service requests. Do you think you are up to the challenge?

Identity Services provide the core services that identify, authenticate and authorize our customers, and provide the information to hundreds of services within the Amazon service oriented architecture. We manage the customer authentication and authorization experience, and are embarking on new and exciting initiatives in this space, both on the web and on mobile devices. If you are excited about solving new business problems using state-of-the-art technologies, and open standards such as OpenId and Oauth, or developing applications and frameworks for mobile platforms, we’d love to talk to you.

Duties will include design, implementation, documentation, support and operations of various facets of Identity Services. We are looking for self-starters who are interested in leading new initiatives. Successful candidates must also be innovative, creative, flexible, self-directed, and able to design and write high-performance, robust and maintainable code. They must have excellent verbal and written communication skills. The ability function at a very high level in a fast paced environment along with a team of very talented engineers is essential.

If you enjoy working in a dynamic environment to deliver world class mission critical systems, this may be the career opportunity for you.

It goes on to say things like they want somebody with a CS degree, and programming experience, and close with “Ability to handle multiple completing priorities in a fast-paced environment.

I can do that. Then came the phone screen. It was a live programming session in a shared document system.

The person on the other end game me the basic problem: You have a list of star coordinates, and you need to find and print out the closest 50 stars to Earth. I asked if I could have a PriorityQueue API, and then was able to complete the exercise with time left over. That was fun.

So, a few days later, I got an email that said, “The hiring team has really enjoyed speaking with you and we sould like to schedule a time for you to come to Amazon for in-person interviews!…” (Yes, they did not put a comma in that sentence.) It went on to describe roles which were really senior, and which I did not actually feel qualified for. The recruiter assured me that not many people outside of Amazon’s core engineering team was qualified when they started, so please come in.

So I flew to Seattle.

It was February, and it had actually snowed the night before. Doesn’t normally snow much there, so there were many traffic problems. The interview was in a tall building downtown. The building had a really great view of the Space Needle, so it had that going for it. I went to the lobby on whatever floor it was. There was a holding area just for interviewees. You had to show ID, and then they printed a visitor’s badge. The entrance to the interview rooms was strictly behind a badge lock.

They took me in. They told me that if I needed to use the bathroom, somebody would have to walk me there and back. Lovely. Felt like a prison.

I had three whiteboard programming problems, and one architecture problem. I would be surprised if my answers to the architecture problem were worth anything; I felt lost.

I don’t remember one of the other problems. However, one of them was a computational geometry problem. I spent 20-25 minutes trying to figure out the definition of a “ray”. It did not help that English was not the interviewer’s first language, and he had a really hard-to-understand accent. That was frustrating.

The other one was this: Given an arbitrary object, write something which would output the object in JSON format. This is not hard, but it is very, very long. I wrote and wrote and wrote and filled three white boards and really did not see the point at all.

That day did not go well.

I had several onsite interviews all bunched together. Resolution of this interview will be in a future post.


Golden Frog – Early 2014

This was my favorite toy as a kid!

When Klink hired me as a contractor, it was supposed to be for 3 months. As the holidays for 2013 were approaching, that three months ran out.

I had a meeting with my boss. He told me that they hired me because the original Mac developer went AWOL one day, and the person that they had wanted to work on the Mac went on family leave, and would be gone three months. She was coming back to work, so they had originally wanted my contract to terminate after 3 months.

(Shudder. I knew that the contract was temporary, but it is still chilling to hear somebody talk about you as a plugable resource.)

However, I had impressed them so much as a Mac developer, that they were going to keep her where she had good skills (backend work), and would like to make a permanent offer for me.

He asked me what I wanted in salary, and I told him, and also said that I wanted Klink to buy me a work machine, since I was using my personal laptop at this point.

Right before the holidays, he presented me a written offer for $5000/year less than the number I had told him. There was no secondary compensation at all. No bonuses; no equity. There was a health plan, and a 401K plan, but still, the offer was pretty disappointing. Not having a job in my back pocket, I accepted, but started working on another position immediately.

As 2014 unfolded, the new Mac for me to work with never developed. And then we lost our lease on our office, because our shadowy corporate masters did not want to sign a 3-5 year lease on a new place. The office moved to a shared office co-op. I asked if I could work at home, since I had worked at home a lot before, and it would mean less space in the shared office. I agreed to be available on Skype, and come in once a week or so for meetings.

So the first part of 2014 was a whirlwind. I am not going to write about chronologically, since everything was happening at the same time. It was hard to keep up with.

One day, I got a phone call for a company looking for a Mac developer. Golden Frog was a local company, and they had a file syncing client to their cloud storage system, much like Klink (or Dropbox or Amazon Drive or Box or Google Drive…) I downloaded the app and took a look.

The app icon was a cartoon version of a Tonka dump truck. It offered what you would expect, but had no “Klink this!” feature, which would establish a link to an arbitrary folder to your sync set. Still, seemed like what I was currently working on. Digging around in their app bundle, I figured out that they were using the same python library to keep track of file system changes.

The recruiter set me up with a phone call with the manager. She described to me the organization and business plan. There were 20 people in the engineering department, all reporting to her. A small number were working on DumpTruck, but the others were working on their proprietary VPN product. In the conversation, she mentioned that the senior engineers were not listening to the product team/upper management, and that she had no visibility to what the QA and release engineering teams actually did.

She also informed me that I was too senior; the budget was for a junior engineer.

I then questioned her:

Me: How many direct reports do you have?

Her: 20

Me: Are there any other managers?

Her: No

Me: What kind of product scheduling methodology do you use?

Her: We are trying to move to Agile, but have some rought patches.

Me: When was the last time anybody took a significant vacation

Her: I don’t know.

I knew I did not want to be an engineer here anyway.

Me: So let me guess as to what is happening here. The engineers are confrontational because they don’t think that they have time to do real work, and there To-Do lists are way too long. QA is complaining because they already have too much to do, but they don’t know what to test with anything new. They are also pushing for automation, but have nobody who can do that work. Release Engineering is pushing for a continuous-delivery system, but since there is no automation, and probably no budget, it is falling on deaf ears. And your management is upset because the team appears to not be delivering anything.

(10 second pause)

Her: Go on.

Me: I am going to boldly assert that I can help you, but as a manager. I have experience managing both engineering and QA teams, and have managed full-cycle development. I have done performance evaluation, product design, and systems design. If you were to hire me, we could work together to make this team much better.

Her: Your guesses as to what is going on around here are scary.


Her: We don’t currently have any budget for another manager, but let me get with my management and talk to them. I’ll get in touch later.

I scared her. I never heard from her again.


New York Yankees – 2014

I mentioned in Project Scoresheet – 1989 that I was an amateur baseball statistical analysis (“stathead”), and dreamed of working in an enlightened front office someday. At time of that post, 1989, there were no front offices truly embracing analytics. Moneyball documented how one club started the process, although Oakland wasn’t alone. By the time the mid 2010’s came along, most clubs were starting to hire data analysts and programmers.

I still get a cold chill when I see this logo, though.

I saw a job listing on some board somewhere for a job as a programmer for some team or another, and when I cliked on the link, it took me to Teamwork Online, which is a site that most of the major league clubs use to post jobs. I signed up for that site, and notifications for analysis and programming jobs.

Over the years, the small handful of positions that were sent to me were not all of that interesting, but one did stand out. The New York Yankees were hiring a mobile developer to work on its internal apps. What the hell? I applied.

A few weeks later, I received the following:

We want to thank you for your interest in the above mentioned position. We had many fine applicants for the position, including you. However, we have decided not to fill this position. We welcome you to apply for any future positions we have available that match your skills and experience.


The Hiring Manager for the “Mobile Application Developer, Baseball Operations – New York Yankees (Bronx, NY)”
MLB Baseball Jobs

Moral of this story: A significant number of job postings disappear before being filled. I do not know if I would enjoyed living in a land which had real winter, but it was slightly disappointing not to be interviewed.

Apple – 2013

Klink was fun. Programming on the Mac was fun. Learning Python was fun. Dealing with the Python-Objective C bridge was not so fun, but you can’t have everything.

However, this was still a contract, and although they had talked about converting to permanent at the end of the contract, I had nothing close to a guarantee. So I had to keep my eye out for another position.

A couple of recruiters contacted me about IOS positions, but they required me to move, or the company looked awful, or any number of other things made those contacts not desirable.

After a few weeks, I got an email from a recruiter at Apple:

I am a recruiter with Apple and recently received a copy of your resume for review.  Would you have interest in speaking with an Engineering Mangers of ours, Dave Chan regarding an opportunity we have for an iOS Performance QA Stress Test Manager? Unfortunately I do not have a job description for you, but Dave can walk you through during your call the role and expectations.  What might work for you this week for a 30 min call?
I look forward to hearing from you.

There were problems with this, of course. It was in Cupertino. It was leading a team of QA and automation people.

I really did not feel like I had a choice but do to pursue this and see where it lead. So, I talked to the hiring manager. It was a non-technical question-and-answer session about my career to date. He suggested I talk to another person, the head of performance engineering. We talked, and had a lively discussion about what performance testing means.

So they decided to fly me out for a full round of interviews. Got an Apple looking email stating “We would like to confirm your interview for the position of IOS Performance QA Stress Test Engineer”. I guess somewhere along the way they decided not to pursue me as manager. Whatever.

I flew out on Sunday for a Monday interview, with my return flight being a red-eye on Monday night.

The interview seemed pretty ordinary, in that, at the time, they only did one technical interview. They had me solve some kind of problem in Python, and I did not do particularly well, considering I had 3 months of experience in Python. Each interview had 2 people, and while that seems normal to me now, it was jarring then.

It was weird being back on the Infinite Loop campus. Parking was almost non-existant. The cafe was very very crowded. One of the strange little things: All of the bathrooms now had the metal wedges bolted to the wall, about 1 meter off of the ground. They were about half a meter long, and they flared out to 20-30 cm on top. People put their laptops, iPads, and notebooks in them while they did their business, and retrieved them after they washed their hands. Wish more corporate bathrooms had them!

I interviewed at the beginning of December, and they kept putting off the decision and putting off the decision, and then they told me I had to wait until after the holidays. Finally, I got this:

Thank you for all of your patience through our process.   I was able to debrief with [Hiring Manager] and unfortunately the team have made the tough decision not to move forward.    While they feel you have some great experience, our role is not the right match for your current background.  I am sorry this role will not work out for you.   Are you currently speaking with any other teams here at Apple?   If not I am more than happy to share your resume to broader recruiting team for review.
I am sorry to not have better news for you and I do wish you all the best.

So. No Apple.



Klink – 2013

It had been two months since Spawn demo’ed the working streaming video game system (with a custom game stick you plug into your TV and a Wi-Fi-based X-box like game controller) to its shadowy corporate overlords at Game Stop. Our lease at our office was up the Monday following the Friday demo, so after the demo was successful, the movers swooped in and packed everything up to move to the new office a couple of miles away.

So, Monday, nothing was setup. The two IT folk started unpacking and setting up the network again. Basic Wi-Fi and DNS was up by Tuesday, but nothing else worked for a week or two.

By the time it did, the office was dead. Game Stop had not had a game plan for a successful demo, and they did not know what to do. They did know, however, that they were not going to build a $100 million data center for us to move into the next stage of testing. Their excuse was that both Playstation 4 and Xbox One were shipping, and all of the cash they had to invest were going into those launches. They were going to try to sell us off to another company.

The game system was brought back up in about 3-4 weeks, so I played some games. By this point, the office had 3-4 people there most of the time….

But 3 more weeks later, I was usually the only one there. Others would drift in and out. And I had nothing to do. I did not enjoy the two games on our system that much, and got bored with them. There was no development work happening. No news on us getting sold…

So I set about to learn IOS programming for real, in the hopes of getting a job as a mobile developer.

At the beginning of August, I received a phone call from a staffing agency that placed technical people as contractors in other companies. The agency paid your salary, and provided something like real benefits after you worked with them for a while. I had a similar arrangement 15 years earlier when I had worked at Sun Labs. It was better than just freelancing in many ways, but you were still hourly and got no paid time off.

Still, they had clients doing IOS and Mac work, and that was interesting to me. It was another attempt to get back into full-time software development.

I had coffee with one of their recruiters at a Starbucks, and after a while, she asked if I would do Mac OS work. Sure! She probed me about me Mac knowledge based on the requirements provided by the client. She then asked for references.

I gave her the name of my friend who referred me to Rock Systems. He told me later that every time she asked about specific knowledge of something, he told her that I was a generalist, and self-motivated, and a quick learner, and that I had a good idea of the complete software development life-cycle.

Right about then, I had a scheduled vacation with my family visiting my mother-in-law out of town. Oxford Global Resources, the staffing company, found a position for me. So, I had a phone conference call with my prospective boss and the chief engineer, and they laid it out.

“Klink Your Life”

Klink was a partnership between a Houston server-based company, called Mezeo, and two Asian telecoms, one in Singapore, and one in the Philippines. They produced a Dropbox-like file syncing client. Two things differentiated them: They could get photos off of flip-phones, and they had a feature where you could sync files on your local hard drive that were not in the special “Klink” folder. The Mac OS engineer they had was going on an extended leave of absence, and they needed somebody to work while they were gone. If things went well, then this might turn into a permanent position.

I talked about my background for a bit, trying to stay away from the fact that I had four months of raw development experience in the last 14 years, and emphasizing my familiarity with the Mac and Cocoa development.

I got back to Austin that week, and went it and talked to my prospective boss at the office. He made me an offer on the spot after we talked. Wanted to know if I could start that afternoon! I told them I need a couple of days, and I would start the following Monday ( 9 days later).

I went into the Spawn office, and the General Manager was there, the one who told me I should brush up my resume four months earlier. I told him I was leaving, and that I was sorry I was only giving three days notice. He laughed and said, “Hey, at least you told me!” and wished me luck.

I thought about just starting the new job and seeing how long it would take Game Stop to fire me while collecting two paychecks. I did not want to risk running  into legal problems with Game Stop, though, so I stayed honest! I wonder how many Spawn people did exactly that, though…

I was back as a software developer, baby!

PS Here are the companies I applied to and got nowhere with during this 2013 job search that I have not mentioned here. Some of them got back to me and we got as far as phone screens; others went nowhere.

  • Apple
  • Tesla
  • General Motors
  • Visa
  • AT&T
  • Advisory Board
  • Google
  • Zenoss
  • New York Yankees
  • Rackspace
  • TRC Entertainment
  • Phunware
  • Bazaarvoice
  • Quantum
  • ADP
  • HomeAway
  • Spanning
  • Spiceworks
  • CA Technologies
  • Dun and Bradstreet
  • SS8
  • Qualcomm
  • EZCorp
  • Yahoo

PPS This product looks a lot like what Spawn was doing: I hope it succeeds; it’s a great idea. Game Stop could have had the world…

Unknown Private Company – 2013

Got a phone call from a recruiter, asking me if I would be interested in a QA/Release/DevOps role. She followed up with an email:

Thank you for taking the time to speak with me. As mentioned, I do have a full-time permanent position available in Miami Beach as a Software Release Manager. The client is a billion-dollar privately held company quick on the rise, they own and operate a conglomerate of websites and really have their hands in everything. I do not have a formal description of what the position will entail, but I have provided below what they are looking for. I will continue to keep you in mind for future opportunities in Texas, do not hesitate to let me know if there is ever anything I can help you with.

Background: working in Linux, Java / PHP / SDLC

SCRUM or Agile

Need to be familiar with packaging and deployment specifically for web-based applications and/or mobile apps/sites.




– Experience with either Jenkins, Bamboo or Hudson

Nice to have:

– Python / JIRA / Confluence

– Knowledge of continuous integration/build environment

I had all of that. Miami? I told her that I was interested, so she called me.

We had a good time talking about servers, release cycles, continuous integration. I thought it was going really well.

And then she said, “One more thing: In the spirit of disclosure, I have to tell you that the client’s business is serving adult content.”

Great. She was recruiting me for a porn site. That’s just what I needed…

I said back, “While I don’t have an overt problem with porn, I don’t want to work in that industry. At the very least, I would not want my family to know, and that might be difficult, given that I would be moving from Austin to Miami. So, I am going to have to pass.”

She said, “I understand. They have been having trouble recruiting because of their business, which is why they hired me. Do you know anybody who might be interested?”

“No, I’m afraid I don’t.”

She sighed, “Well, thanks for talking to me, and I wish you luck.”

There is some 12-year old part of me that wanted to see what would have happened, but I am glad I did not pursue it further.