How not to spend eleven months searching a job in Munich

There's a Python user group, which is actually more a Plone user group, in Munich, which was meeting regularly for more than a decade, long before Meetup.com arrived and scaled the technology meetings scene up by a couple of orders of magnitude. I'm frequenting their meetings irregularly since five years, they are my favourite technology bunch in Munich. Following conversation took place there about two weeks ago.

Where are the Python jobs?!?

In a meetup at the beginning of November two of the veterans, Philip and Alex, were talking about recent conferences to an audience of mostly politely silent young people; after the presentation, we were asked if there were any questions.

There've been some; among other, Mike (name changed) asked something like: "where do you find Python jobs in Munich"? And stated he wants to specialize in Python, but there seem not to be many Python jobs around. In Munich, which is supposed to be a a high-tech center, everything is here, you know, BMW, MAN, Siemens, Google, Microsoft, Amazon, and Audi around the corner, just to name the best known, plus a long list of software-we-do-anything-you-wish-if-you-can-afford-it-agencies, plus many many startups, and I've even ignored huge ranges to keep this article short.

Alex answered naming a couple of the usual suspects among the local companies, noting that data science is what drives the current Python boom, but there've been areas where it was traditionally used for a long time, like web development; I added the situation actually got better in the last two or three years, now you can find a company hiring a Python programmer or two any time of the year, while way back there have been times you'd see announcements requiring Python just once in a while...

Alex contemplated it's a bit of a chicken-and-egg-problem, as Munich higher education schools still use Java as their main language to teach programming, because there are still more Java jobs here, and companies don't use Python as they can't find enough hands to do the job. He also mentioned a technology startup, rather prominent in Munich, that was not able to find enough Pythonistas so they switched to Java after a couple of years...

Soon after that we moved from the classroom at Ludwig Maximilian University to our favourite italian restaurant (owned by a greek gentleman) to keep the conversation up over some beer and pasta.

About Mike

Now later in the conversation it turned out that Mike was not a rookie, he'd been programming since he was ten or so, has got a degree from Carnegie Mellon University (which, I was told, is regularly ranked No. 1 in comparisons of computer science degrees in the USA), and has been working for six years since then. Nevertheless, last time he'd searched a job in Munich it took him eleven months sending out applications to land one. Most of the applications were never answered, it was more like shouting into the dark without an echo. He was worried it could be a long search this time, too.

About Munich

His story reminded me of my beginnings in Munich in 1994. While I was studying in Belgrade we were often told that our University has a good reputation in international rankings, especially in the USA; but, like Mike, here I found that nobody knew, and nobody cared. International students and professionals used to be rare. I just deleted a couple of passages giving more detail, as I found I got depressed after rereading them. ;)

Things have gradually improved a lot for foreign students and young professionals since then (!), but it's still true: Munich doesn't have a culture of welcoming the ambitious and hardworking academics. You here? Well OK, as long as you don't bother. There's a wast range of exceptions to the attitude depicted in the previous sentence, but still: if you don't have a job contract before arriving in Munich, expect no hugs on arrival.

What can Mike do about it?

While Python might be less visible in job announcements, the truth is: Python's everywhere these days. I think that Alex brought up the notion of hidden jobs and communities. I was able to confirm. I spent quite some time building different testing frameworks in the german automotive industry. Most of them were either Python-based or at least included Python in some capacity. It's just huge there, and you can't really tell by looking from the outside, because, often, when they're looking for somebody to do the job, they require experience in the darn testing framework, that nobody has ever heard of (except for the people who built it or are forced to deal with it), instead of naming Python which would be perfectly sufficient to solve the problem. How comes? Well the people who write job announcements are often not the ones who really understand what's needed to get the job done. They are HR or recruiters; their job is most of the time not to find somebody for the job, but to filter out all the candidates that don't fit, before they present a shortlist to the technically knowledgeable project leader/manager. And filter they do. (Well obviously not all HR or all recruiters; but I've met my fair share of those who didn't understand the words they were writing).

(The other "hidden" job fields mentioned were security and penetration testing, dev ops, testing in medical devices, bioinformatics / computational research in natural ans social sciences... Both Alex and I had heard people were actively using Python in those fields in and around Munich, but had neither first hand experiences nor detailed information).

And the other thing is: if you go the official way of answering to a job announcement, more likely then not your resume is one of ten that were presented to the manager (out of fifty or two hundred, who never made it to the next round). The manager, being a busy person, chooses to talk to maybe three people with the best looking resumes. Unless there is something clearly exceptional in them, he'll choose those which were well written. So, at this point of time your skill in in writing a resume is far more important than your capacity to get the job done...

So what's the solution? Skip most of the cycle altogether. Try to have a direct conversation to the person with the job before they even write the job description.

How do you do that?

The best way to do so is to be so well known in the field / have such a strong reputation that people contact you when there's a job to be done; but this assumes a long strategical career building.

The second best way is to get recommended for the job by someone who knows or assumes you can get it done. This requires some kind of network, which can possibly be created much faster than the previous, if you have good, or at least some, people skills. (This is, btw, an option Mike applied with at least some success; I forwarded him an project offer I got, but had no time to serve, and he got an interview... result unknown at the time of writing).

The third best way is wide open to everyone. Nowadays companies organize masses of technical meetups and presentations; go there and talk to the people! Do your homework, try to find out in advance who's in charge and what they need. Have a (possibly short, bur relevant) resume with you. If you're prepared they'll likely be happy to talk to you! Use the conversation to find out even more specifically if they're looking for something. Maybe you won't leave the resume right there, but go home to rework it for relevant detail. If you do this three to five times, you'll more likely than not have an interview, and it could be the only interview they do, and not one out of five or ten.

The last good option I know: find (somehow) the relevant persons - via internet or by asking people you know. Then grab the phone - the phone is your friend. Writing unsolicited emails or letters is much more tedious - and who even reads email these days anymore?

Last but not lest, additionally to anything previously said: it might sound obvious, but if you're in Munich / Germany / anywhere outside your natural habitat, and plan to live and work there for anything more than a few days - learn the language of your environment as well as you can!

Path autocompletion in the PyCharm IPython console: a story of suspense

Disclaimer I: I love PyCharm; bellow just happens to be the description of one of its darker corners.

Disclaimer II: The whole post might be rendered obsolete any time soon (and I hope it will). It is relevant, though, at the time of writing, with PyCharm Community Edition 2017.2.3 and with Anaconda 5.0 / Python 2.7.13 and Anaconda 5.0.1 / Python 3.6 and below, on several different versions of Windows 7 (Home, Professional etc.).

How to fix path autocompletion in the PyCharm IPython console

There's a couple of questions scattered around the net related to path autocompletion in the PyCharm IPython console... but the useful answers are rare, almost non-existent, and scattered around as well.

So, what's there to fix after all?

Well for example, when I type:

c:/U<TAB>

I'd expect autocompletion to offer a sensible list of directories and files in folder c:/, starting with U.

On my computer this happens to be just c:/Users.

Yet what autocompletion proposes looks like a random list of keywords and variables from some Python namespace:

/images/nonsense.png

WTF?!

To make this even more weird, "my" IPython (which is part of an Anaconda installation), used in the Anaconda prompt, merrily autocompletes path items "all tha way down":

/images/better.png

... and c:/U<TAB> directly resolves to c:/Users.

WTF square. Isn't this the same software here, and if so, how comes it behaves differently?

I found a way to tame autocompletion in a manner good enough for my situation, and I'm just about to describe it. I should probably just mention I'm neither a specialist on the IPython API nor a champion of PyCharm configuration, so there are certainly better, more thorough ways to do this.

For the impatient: A quick fix in code

If you add this code:

# Completer, available to be imported in starting script
def create_my_file_list(self, event):
    my_files_list = ["c:/Path_A", "c:/Path_B", "c:/File_C"]
    return my_files_list


ip = get_ipython()  # Here we get access to IPython
ip.set_hook('complete_command', create_my_file_list, re_key='.*mymagic')

from IPython.core.magic import register_line_magic
@register_line_magic
def mymagic_some_functionallity(file_name):
    print "they asked me to print", file_name

to the Starting script (File->Settings->Build, Execution, Deployment->Console->Python Console) for the PyCharm IPython console, and get a new console window - or restart the console, writing:

%mymagic_some_functionallity c<TAB>

will offer:

/images/a_fix.png

Now, what do we have here?

The funtion/method create_my_file_list is a 'completer' in the vocabulary of the IPython API. It returns a list of (in this case, fictional) paths.

When added to the IPython-machinery with the call:

ip.set_hook('complete_command', create_my_file_list, re_key='.*mymagic')

the completer will provide autocompletion proposals if autocompletion is initiated in a line matching the regular expression '.*mymagic'.

The name of the self-written magic command mymagic_some_functionallity matches the regular expression above.

So when %mymagic_some_functionallity is at the beginning of the line, the list of autocompletion-proposals provided by the completer create_my_file_list is right at the top of the proposal list.

For the why-s that lead to the code above and the caveats, keep reading.

What is the reasoning that lead to the code above?

My premise to start with: I needed a way to feed my new magic commands with some paths as entry parameters, and it had to be more elegant than copy-and-paste, or typing the whole path.

I soon realised drag&drop will not work with PyCharm, so I went for the second best: autocompletion. Which also presented itself dysfunctional, in the way described above.

Knowing nothing about IPython configuration, I started googling and leaving questions at Stack Overflow, Reddit, the IPython-dev-list, PyCharm support site, all without much reaction.

In those places asked where I can find out about how to influence PyCharm IPython console autocompletion behaviour, any starting points?

I got no answers, but, turned out, a good starting point is the IPython documentation , including the IPython API documentation. No, I didn't try to learn all of the IPython API, but reading the documentation I learned there is something called completers, and that you can write custom ones.

A search for completers lead to a blog post describing how to write custom completers, which contains the essence of the solution above.

Some things to notice:
  • The form of the call to a completer function is def name_of_the_completer(self, event).
  • The parameter self is obviously in order for the completer to be added to some class (by calling ip.set_hook).
  • The parameter event defines when the code of the completer will take effect, e.g. be called. I our case this is when the beginning of the IPython command line corresponds to a certain regular expression.
  • In order to call ip.set_hook we need a code handle to the current IPython instance, ip. with the line ip = get_ipython()

Last thing to mention: obviously, knowing what you know now, the list of paths you might want to offer in the completer is up to you. I had a very specific problem to solve, so my choice is irrelevant here. But the principles presented apply nevertheless.

Why is the completer matched just by a specific line pattern?

Now, theoretically we could have written a completer matching any form of line beginning, instead of just matching our custom magic commands.

It just happens to be that adding our completer the way above will overwrite the addition of magic commands to the IPython autocompletion machinery, so this was not an option to me.

And the reason that adding our completer will overwrite magic commands autocompletion? I this document, a change proposal stating that the IPython tab completion machinery is messy, complicated, and difficult to extend, there's a section Limitations of the Current System stating "the fact remains that the custom completers are always exclusive-- once one returns sucessfully, its results are immediately sent to the frontend and no merging of completion lists is performed". Please consult it, if you want to dig deeper; it's probably a good starting point,.

How comes IPython behaves differently in Anaconda prompt and in PyCharm?

Last but not least: IPython behaves differently in Anaconda prompt and PyCharm - how comes?

After some banging-had-to-wall: while it's the same IPython, it does not use the same configuration / setup.

Ah, the pleasures of introspection!

In PyCharm:

In[5]: get_ipython
Out[5]: <bound method PyDevTerminalInteractiveShell.get_ipython of <_pydev_bundle.pydev_ipython_console_011.PyDevTerminalInteractiveShell object at 0x0000000003D73940>>

On my computer, the _pydev_bundle resides in C:\Program Files\JetBrains\PyCharm Community Edition 2017.2.3\helpers\pydev\_pydev_bundle.

In Anaconda prompt:

In [3]: get_ipython
Out[3]: <bound method TerminalInteractiveShell.get_ipython of <IPython.terminal.interactiveshell.TerminalInteractiveShell object at 0x0000000003B80198>>

Obviously, PyCharm forked its IPython configuration from PyDev, and it's not using the configuration machinery in Anaconda. That's the reason they don't behave the same.