Klink was fun. Programming on the Mac was fun. Learning Python was fun. Dealing with the Python-Objective C bridge was not so fun, but you can’t have everything.
However, this was still a contract, and although they had talked about converting to permanent at the end of the contract, I had nothing close to a guarantee. So I had to keep my eye out for another position.
A couple of recruiters contacted me about IOS positions, but they required me to move, or the company looked awful, or any number of other things made those contacts not desirable.
After a few weeks, I got an email from a recruiter at Apple:
I am a recruiter with Apple and recently received a copy of your resume for review. Would you have interest in speaking with an Engineering Mangers of ours, Dave Chan regarding an opportunity we have for an iOS Performance QA Stress Test Manager? Unfortunately I do not have a job description for you, but Dave can walk you through during your call the role and expectations. What might work for you this week for a 30 min call?
I look forward to hearing from you.
There were problems with this, of course. It was in Cupertino. It was leading a team of QA and automation people.
I really did not feel like I had a choice but do to pursue this and see where it lead. So, I talked to the hiring manager. It was a non-technical question-and-answer session about my career to date. He suggested I talk to another person, the head of performance engineering. We talked, and had a lively discussion about what performance testing means.
So they decided to fly me out for a full round of interviews. Got an Apple looking email stating “We would like to confirm your interview for the position of IOS Performance QA Stress Test Engineer”. I guess somewhere along the way they decided not to pursue me as manager. Whatever.
I flew out on Sunday for a Monday interview, with my return flight being a red-eye on Monday night.
The interview seemed pretty ordinary, in that, at the time, they only did one technical interview. They had me solve some kind of problem in Python, and I did not do particularly well, considering I had 3 months of experience in Python. Each interview had 2 people, and while that seems normal to me now, it was jarring then.
It was weird being back on the Infinite Loop campus. Parking was almost non-existant. The cafe was very very crowded. One of the strange little things: All of the bathrooms now had the metal wedges bolted to the wall, about 1 meter off of the ground. They were about half a meter long, and they flared out to 20-30 cm on top. People put their laptops, iPads, and notebooks in them while they did their business, and retrieved them after they washed their hands. Wish more corporate bathrooms had them!
I interviewed at the beginning of December, and they kept putting off the decision and putting off the decision, and then they told me I had to wait until after the holidays. Finally, I got this:
Thank you for all of your patience through our process. I was able to debrief with [Hiring Manager] and unfortunately the team have made the tough decision not to move forward. While they feel you have some great experience, our role is not the right match for your current background. I am sorry this role will not work out for you. Are you currently speaking with any other teams here at Apple? If not I am more than happy to share your resume to broader recruiting team for review.
I am sorry to not have better news for you and I do wish you all the best.
It had been two months since Spawn demo’ed the working streaming video game system (with a custom game stick you plug into your TV and a Wi-Fi-based X-box like game controller) to its shadowy corporate overlords at Game Stop. Our lease at our office was up the Monday following the Friday demo, so after the demo was successful, the movers swooped in and packed everything up to move to the new office a couple of miles away.
So, Monday, nothing was setup. The two IT folk started unpacking and setting up the network again. Basic Wi-Fi and DNS was up by Tuesday, but nothing else worked for a week or two.
By the time it did, the office was dead. Game Stop had not had a game plan for a successful demo, and they did not know what to do. They did know, however, that they were not going to build a $100 million data center for us to move into the next stage of testing. Their excuse was that both Playstation 4 and Xbox One were shipping, and all of the cash they had to invest were going into those launches. They were going to try to sell us off to another company.
The game system was brought back up in about 3-4 weeks, so I played some games. By this point, the office had 3-4 people there most of the time….
But 3 more weeks later, I was usually the only one there. Others would drift in and out. And I had nothing to do. I did not enjoy the two games on our system that much, and got bored with them. There was no development work happening. No news on us getting sold…
So I set about to learn IOS programming for real, in the hopes of getting a job as a mobile developer.
At the beginning of August, I received a phone call from a staffing agency that placed technical people as contractors in other companies. The agency paid your salary, and provided something like real benefits after you worked with them for a while. I had a similar arrangement 15 years earlier when I had worked at Sun Labs. It was better than just freelancing in many ways, but you were still hourly and got no paid time off.
Still, they had clients doing IOS and Mac work, and that was interesting to me. It was another attempt to get back into full-time software development.
I had coffee with one of their recruiters at a Starbucks, and after a while, she asked if I would do Mac OS work. Sure! She probed me about me Mac knowledge based on the requirements provided by the client. She then asked for references.
I gave her the name of my friend who referred me to Rock Systems. He told me later that every time she asked about specific knowledge of something, he told her that I was a generalist, and self-motivated, and a quick learner, and that I had a good idea of the complete software development life-cycle.
Right about then, I had a scheduled vacation with my family visiting my mother-in-law out of town. Oxford Global Resources, the staffing company, found a position for me. So, I had a phone conference call with my prospective boss and the chief engineer, and they laid it out.
Klink was a partnership between a Houston server-based company, called Mezeo, and two Asian telecoms, one in Singapore, and one in the Philippines. They produced a Dropbox-like file syncing client. Two things differentiated them: They could get photos off of flip-phones, and they had a feature where you could sync files on your local hard drive that were not in the special “Klink” folder. The Mac OS engineer they had was going on an extended leave of absence, and they needed somebody to work while they were gone. If things went well, then this might turn into a permanent position.
I talked about my background for a bit, trying to stay away from the fact that I had four months of raw development experience in the last 14 years, and emphasizing my familiarity with the Mac and Cocoa development.
I got back to Austin that week, and went it and talked to my prospective boss at the office. He made me an offer on the spot after we talked. Wanted to know if I could start that afternoon! I told them I need a couple of days, and I would start the following Monday ( 9 days later).
I went into the Spawn office, and the General Manager was there, the one who told me I should brush up my resume four months earlier. I told him I was leaving, and that I was sorry I was only giving three days notice. He laughed and said, “Hey, at least you told me!” and wished me luck.
I thought about just starting the new job and seeing how long it would take Game Stop to fire me while collecting two paychecks. I did not want to risk running into legal problems with Game Stop, though, so I stayed honest! I wonder how many Spawn people did exactly that, though…
I was back as a software developer, baby!
PS Here are the companies I applied to and got nowhere with during this 2013 job search that I have not mentioned here. Some of them got back to me and we got as far as phone screens; others went nowhere.
Got a phone call from a recruiter, asking me if I would be interested in a QA/Release/DevOps role. She followed up with an email:
Thank you for taking the time to speak with me. As mentioned, I do have a full-time permanent position available in Miami Beach as a Software Release Manager. The client is a billion-dollar privately held company quick on the rise, they own and operate a conglomerate of websites and really have their hands in everything. I do not have a formal description of what the position will entail, but I have provided below what they are looking for. I will continue to keep you in mind for future opportunities in Texas, do not hesitate to let me know if there is ever anything I can help you with.
Background: working in Linux, Java / PHP / SDLC
SCRUM or Agile
Need to be familiar with packaging and deployment specifically for web-based applications and/or mobile apps/sites.
– Experience with either Jenkins, Bamboo or Hudson
Nice to have:
– Python / JIRA / Confluence
– Knowledge of continuous integration/build environment
I had all of that. Miami? I told her that I was interested, so she called me.
We had a good time talking about servers, release cycles, continuous integration. I thought it was going really well.
And then she said, “One more thing: In the spirit of disclosure, I have to tell you that the client’s business is serving adult content.”
Great. She was recruiting me for a porn site. That’s just what I needed…
I said back, “While I don’t have an overt problem with porn, I don’t want to work in that industry. At the very least, I would not want my family to know, and that might be difficult, given that I would be moving from Austin to Miami. So, I am going to have to pass.”
She said, “I understand. They have been having trouble recruiting because of their business, which is why they hired me. Do you know anybody who might be interested?”
“No, I’m afraid I don’t.”
She sighed, “Well, thanks for talking to me, and I wish you luck.”
There is some 12-year old part of me that wanted to see what would have happened, but I am glad I did not pursue it further.
The primary job site I used for my search in 2013 was Dice.com. Around the beginning of July, I heard from a recruiter on the phone. I sent him my resume, and he sent back the following:
Subject: Software Development Ops engineer – Full time position with Polycom in Austin, TX
I saw your resume in DICE and would like to reach out to you for a full time position with Polycom for above mentioned job opportunity in Austin, TX. Job description and requirements are attached with this email. Please review these requirements and if interested, please send me your resume in WORD format. Once I get your resume, I will contact you to discuss additional information about this position. If you have any questions, please feel free send me an email or call me.
Thank you and have a good day
This position was to assume a “Dev Ops” role. Basically, release engineering, and QA, and keeping online systems up and running. The desired experience including building software for Linux, Windows, Android, and Embedded Linux, and deep familiarity with both git and subversion. Right up my alley. There were other things listed; some I had, but some I did not, but they were not as important.
So I sent him my resume.
The hiring manager called me, and he described a set of mature projects, both desktop and web, and some new projects, mostly embedded. I talked about release engineering, ant, make, yocto, repo, Jenkins, git, subversion, maven, etc. At the end of the conversation, he told me that wanted to bring me onsite.
I talked to three engineers:
1. Software engineer on the embedded product. Basically, they were developing a new video conferencing device, with USB and HDMI inputs. Custom Android kernel. Some custom apps. I talked about how I would setup a continuous delivery system for that. At the end of this interview, he said “We’ve got to get you hired here!”
2. The hiring manager was second. We talked more about general software development practices, agile, scrum. We also talked about performance reviews and hiring, which was a little odd. He told me, “Well, I hope the others like you, because I want to start the process of getting you an offer right now.”
3. The last person I talked to is the person I was to replace; he was taking a role as a full-time software developer. We talked about everything he was currently doing, and I asked questions, and had some suggestions. About every five minutes, he said, “Wow! Where did they find you?” or “Man, we’ve got to have you.”
At one point, he was describing their new video conferencing device that had a custom Android kernel. I asked what the processor chip was, and he said it was an OMAP chip. Just so happens, that was the same chip that Spawn Labs was using in its game stick!
So, I asked, “Did you get source to the video driver? That was a pain.”
He said, “Oh, yeah, that was. We had a hell of time figuring out why it would not load.”
So I described how the Linux kernel would not load drivers from modified Git repos if it itself was built with an unmodified repo, and how I patched the kernel to load the specific driver as an exception to get things running.
He blinked, and paused, and said, “We have to make you an offer.”
I went home feeling pretty good. The hiring manager had not blinked at the salary range I asked for, and I thought I had a good chance.
What that meant to me was they immediately froze all hiring. The offer had been written, and was waiting for an executive signature. But that did not happen.
After I got official confirmation that this was not going to happen, I sent this to the fellow whom I was to backfill for:
I just found out that Polycom has a hiring freeze and that I won’t get
getting an offer from them. That’s too bad for me, but that’s life.
However, I wanted to give you some advice.
The hardest thing to do professionally is change positions within a
company, particularly one that is big and unwieldy like Polycom appears
to be. My advice to you is: Get out. Find a position somewhere else. You
are in a rut there, and your last best opportunity that Polycom had to
offer is now closed to you. Leave. That is the only way you will advance
your career in a direction you want to go.
I am not saying this because I think that they would back-fill your
position and make me an offer. They won’t. Not anytime soon.
If you want, we could get together for coffee. Or not. Up to you. Good
luck to you, and thanks for making the effort to talk to me.
He wrote back:
I’m glad the HR and recruitment people did their jobs and notified you
that the req was canceled. Sometimes that doesn’t happen. We also had a
layoff a couple of days ago, but I don’t think anyone in Austin was
I do appreciate that you thought of me and recognized the rut I’m in.
You’re definitely right that it would be best for my career and job
satisfaction to leave. However, my plan is to stick around for two or
three more years. I’m taking advantage of Polycom’s above market salary
to reach a personal retirement milestone. After that, I’ll feel much
more free to work somewhere else at market rates or even below market
rates if necessary. I continually reevaluate that decision, but for now
it still feels like the right answer. Thanks again for thinking of me.
I talked to a variety of companies after Spawn sputtered out. Most of them were unforgettable phone screens, after which about half of them ghosted me. That’s the way it goes.
I did get called by a recruiter. She had a Release Engineer position in Knoxville, TN. My mother-in-law lives in Kentucky, and having my family closer to her would have been great. So I requested more information.
Here is what the blurb said about the company:
Scripps Networks Interactive is one of the leading developers of lifestyle-oriented content for television and the Internet, where on-air programming is complemented with online video, social media areas and e-commerce components on companion Web sites and broadband vertical channels. The company’s media portfolio includes Lifestyle Media, which is comprised of popular lifestyle television and Internet brands HGTV, Food Network, Travel Channel, DIY Network, country music network Great American Country, and the Fine Living Network, soon to be rebranded as the Cooking Channel; and Interactive Services, with leading online search and comparison shopping services BizRate, Shopzilla and beso.
So, if they relocated our family, would we have to do House Hunters?
Anyway, the job was a typical release engineering position for the time. Lots of building automated infrastructure for builds, moving to continuous integration. Not exactly what I was looking for (I wanted to code), but it would be better than nothing.
However, after I discussed this with my family, we decided the only way I would take this job would be if they paid really well.
Spawn was much more ambitious than I originally thought. In building a streaming game service, we were building:
A board with 4 CPUs on it. This was divided into two groups of two, consisting of:
An embedded Windows computer (Application Program Unit) – This computer actually ran the game itself
An embedded Linux Computer (MPU – don’t remember what ‘M’ stood for) – This computer controlled all input and output of the Windows machine, as well as its network stack. It handled messaging from the Internet, simulated a game controller, etc.
A cabinet with 20 of these boards
Handled the network routing of the 40 computers inside
A server farm in the data center to provide disk space to hold the games themselves and the users’ home directories for saving games.
A Windows application which would connect to our service. The user would plug a game controller into the computer, and the application would translated that into network commands that would talk to an MPU. It would also have an interface to selecting games.
A Mac application (never got started) to connect.
An Android Java application that could connect directly to our service.
A game stick (like a Roku, Firestick, or Chromecast). This device would plug directly into the TV’s HDMI port, and use USB for power (and for diagnostics or software updates). It had buildin WiFi and Bluetooth stacks for controllers to connect to.
A remote control for the initial setup of the game stick.
A custom game controller designed for connecting to our game stick via WiFi or Bluetooth.
We wanted to build out 5 data centers in North America, with more around the world as we grew.
When I got there, the first prototypes of everything were being developed. I was QA and Release Manager, but had no staff, so the first thing I worked on was building all of this software. I had to learn how we built:
A Windows application (easy)
Software for the MPU (embedded Linux)
Software for the game stick (Building an Android OS)
An Android application
Software for the remote control
Building for the Game Stick was the hardest to build, and I actually had to learn how to debug the Linux Android kernel on the device itself to figure out what our build problems were (1).
It was bold. It was ambitious. It was expensive.
I started in September, and by January things appeared to be coming together. And then… our chief embedded engineer left. And other people were starting to leave. My boss cancelled my weekly one-on-one, and never bothered to reschedule it.
A few weeks later, he quit, announcing that he was moving his family to Hollywood so he could help his daughter’s acting/modeling career.
It was obvious we were in disarray. About that time, the General Manager called me in to his office, and told me, “I can’t promise you that you will have a job going forward. If I were you, I would polish your resume, and start looking. I would be happy to be a reference, and I am fighting to keep us going. But I have to be realistic.”
Huh. Well, that sucked.
About a week later, he announced that Game Stop, our parent company, had given us a deadline. We had to show whatever we had in June.
And that team pulled together. Everybody worked hard on building a tiny data center with one board (two nodes) that could play from a catalog of two games. We hand-built two game sticks, and got two controllers to work with them. I was left with not much to do because we were in demo/hacking mode, but the last week leading up to the demo, I tested the hell out of the system. It worked great, and I got to play Borderlands 2 for a while. While we were putting this together, though, we were informed that our lease was running out, and the last day in the office that we had been at was going to be Demo Day.
On that day, our execs went up to Game Stop headquarters in Dallas, while the rest of us waited to tackle any fires that came up.
The demo went flawlessly. None of the Game Stop execs could believe it. They took the game sticks from office to office, plugged them in, and played games. They could not make the system fail. The saved game worked flawlessly. The video was good. The audio was good. There was no perceptible lag. It could not have gone better.
This was not expected. They honestly thought we were going to fail, and from what I understood, had been ready to lay us off the next week. But evidently, there was some sort of contractual thing with this demo, and they had to keep at least the management of Spawn around because we delivered on our promises.
When asked if we could start the next phase, which was to actually build a real data center in Dallas for a “regional beta”, they told us that we were going to be delayed; Game Stop had to deal with the rollout of both PlayStation 4 and Xbox One, and that they did not have the capital to build out a hundred-million-dollar data center.
Meanwhile, back at the ranch, as soon as the demo was over, everybody packed everything up. The IT people were up for almost three days; they got everything done in time for the trucks to move all of it to the new office on Sunday.
When we got to the new office on Monday, nothing was up at all. We all started helping the IT people start to build out the network and the internal infrastructure, but there was only so much the rest of us could do. Once there was WiFi, I was there to help with lifting and sorting cables, but the rest of the time, I had nothing to do at all.
And that did not change. Game Stop started trying to shop us around to other companies, but I don’t think the world was quite ready for us. Google Stadia was years away, and Game Stop was worried that Steam was going to start streaming games. Nobody bit.
I still continued to go to the office, but there was no work to do at all. So I spent my time learning more about programming for IOS. Was very productive! But I had to start looking for jobs.
I don’t know how the initial contact happened for 21CT, but I got a job listing for a Technical Lead in my email. It looked much more like a Project Manager position than anything else. I did a phone screen (nothing remarkable there). After the phone screen, they called to setup a day of interviewing. Looking good…
… and the day before the interview, they got in touch via email and told me that they had had a restructuring, and that the position was no longer open.
I’ve heard that before. Sigh.
(1)Basically, the 3rd party developer left instructions on how to patch the software, build it, and deploy it. But I wanted to put the finished source code after patching into source control. We did not have access to the source code of the video driver we were using, which is annoying. When I built the system via patching, it booted fine. But when I put in source control, it crashed on launch. Turns out, the video driver was built with a source code control set of source that was modified. When I build the Linux kernel with patches, it was also built with source that was modified. When I pushed into source control, it was not, and the kernel refused to load drivers that were built from modified source if it itself was built with unmodified source.
I don’t normally talk about my work as a part-time musician, and it normally can be treated just as a hobby.
In March, I got contacted to play clarinet and bassoon in Zilker Hillside Theater‘s production of Sound of Music. The productions run six weeks, the last half of summer before the school year starts again. There are shows Thursday-Sunday evenings starting at sunset. The seats are the hillside looking at the stage, which is across the parking lot from Barton Springs Pool. It’s usually hot, but being staged mostly in the dark helps a lot.
Best thing is, it pays. Oh, not a huge amount, but fairly well for music.
After I accepted the show, I got laid off from Coverity, and had done a consulting gig, but that wrapped up just as the show was starting. That meant I actually had time to practice, which, considering I am not world’s best bassoon player, was great!
And then I got my $1600 check from Zilker. Hallelujah! It was the only time as an adult that my music habit supported me and my family. Without that gig, our mortgage payment would have been late for the only time in my life.
I played the 2013 show, also, and spot subbed a couple of other times, but there are different directors now so I don’t get called anymore. But Zilker sure saved us in 2012!