.. title: Why labautomation .. slug: why-labautomation .. date: 2017-10-05 09:54:34 UTC+02:00 .. tags: labautomation .. category: .. link: .. description: .. type: text When I began my studies for a PhD in Physics I like many other graduate students had some basic programming skills mainly based around Matlab and a little C. Most of this I learned through beginners courses during my undergraduate studies and the need for some basic numerical analysis for my Masters thesis. At the start of my PhD I felt simultaneously excited and was eager to start and get into the laboratory, and overwhelmed with the what I needed to do. I dove into the literature, reading about my research topic, began implementing some simulations in C, and started some initial experiments. Like many others, my first measurements were essentially manual, adjusting my equipment by hand saving data, such as optical spectrum analyser traces to floppy disks and writing measurement results into my logbook. When I used computers to control equipment, it was typically based on small scripts inherited from previous students written in labview or the matlab instrumentation toolbox. While this approach worked fine initially, as my experiments became increasingly sophisticated I began to realise that I needed a more systematic approach for conducting my measurements. While I greatly enjoyed planning, designing and setting up my experiments, sitting in the lab for hours on end, pushing one or two buttons and noting values in the logbook was not fun, quite apart from the danger of making avoidable mistakes that more than once made me need to retake hours of measurements. At the same time I had found the Python language for some hobby projects around processing web-pages. I am a fan of free software, and the open source nature of the language strongly appealed to me. Moreover, the structure of Python, which makes extended use of whitespace, and the very natural syntax allowed me to learn quickly. After a short time I was able to write my own programs and read other projects code which is a great source of learning. I learned about the powerful scientific programming packages such as numpy, scipy and matplotlib and the growing scientific community that uses Python for their simulations, analysis and experimental control. I was convinced and decided to use Python as the major language to automate my experiments. After a short time I had created fully automated experiments that ran unsupervised allowing me to concentrate on the fun parts of an experimental PhD instead of sitting for hours in the lab taking measurements. I have graduated a long time ago, but I continue to enjoy programming in Python and use it almost exclusively for my research work. In 2015 Nicolas Fontaine from Bell labs, Binbin Guan from Acacia and myself now a tenured Senior Researcher at Chalmers University of Technology, all big open source and Python fans, decided to share our experiences with others, in particular students and early career researchers, so that they could learn from our mistakes and to bring together the community of open source and Python users in photonics, we knew existed. The labautomation hackathons were born. While the labautomation hackathons are generally language agnostic, we place a strong emphasis on using Python. In our view Python has some strong advantages over other languages: 1. It is a programming language first with a large user base in areas ranging from web programming to automated system administration tools and desktop applications such as video editors. Therefore Python really shines when integrating non-scientific tools into your programs. For example the packages for writing graphical user interfaces (GUIs) are advanced and sophisticated. Unlike in other languages which developed out of scientific tools, advanced programming concepts, such as object oriented constructs, do not feel bolted on. 2. Because Python is so widely used throughout many different industries – e.g. much of Googles internal tools are based on Python, and dropbox infrastructure is mainly based on Python – Python proficiency is a great skill for students CV, and I know of several postgraduate students who moved on to careers outside of photonics based around Python programming. 3. The scientific community around Python is growing at breathtaking pace, and new sophisticated tools are being released every week. Some examples are the pandas and jupyter packages which are used extensively in data science, which is becoming increasingly dominated by Python. It happens often that if one encounters a problem, we can find someone who has already solved it and released the code under a permissive open source licence. 4. Python’s syntax, use of whitespace, and philosophy that there should only be one way of doing things, forces Python code to generally be very easy to read and understand. While it is possible to write complex, opaque and intelligible code in Python it is generally much more difficult. This is particularly advantages for scientific programming, where we often write code which gets used extensively, but we only come back very sporadically to make changes. 5. Python and all the main scientific packages are open source and therefore free to use. Therefore programming in Python does not incur the high licence costs, that can be prohibitive in particular for small businesses or start-ups. We held the first labautomation hackathon on Sunday evening at the Optical Fiber Communication Conference (OFC) in March 2017. The speakers at the first event were Nicolas Fontaine from Bell Labs who gave a short introduction to the event and some basic tips on using Python. I was talking about using the bokeh package to do interactive plotting inside the browser. Ryan Scott from Keysight shared his experiences of switching his labautomation from labview to python and the reasons why he did it. Jorris Geesels and Piere Wahl from Luceda Photonics introduced the IPKISS photonic design software which is written in Python. Finally we had a live demo organised by Binbin Guan from Acacia, where participants could adjust the different parameters of a simulated IQ modulator running on a web server. Many lively discussions developed from the talks, with the less experiences of the about 50 participants asking questions about how to get started or what packages to use, and some other participants sharing their experiences and reasons why they had already made the switch. The discussions continued well into the evening when drinks and food were served thanks to the generous sponsorship from the optical society of America (OSA). After the great success of the first hackathon we decided to continue the event and held the second hackathon at the European Conference on Optical Communication (ECOC) in September in Gothenburg, Sweden. With 100 pre-registered participants the event looked to be even bigger than the OFC event. However, because the welcome reception which was running at the same time and the difficult to find location of the room, it meant that about 50 of the participants showed up to the hackathon. This second event was strongly focused around industry use cases of Photonics, prompted by many questions to this extend at the OFC hackathon. The speakers at this event were: 1. Ronald Broeke from Bright Photonics who spoke about open source photonic IC design with Nazca 2. Pieter Dumon from Luceda Photonics on regression testing of component libraries and designs 3. Jeff Dralla and Rolf Madsen from Keysight talking about Python Automation for Test & Measurement with TAP 4. Andre Richter and Sergei Mingaleev from VPI Photonics who introduced their Python-based IDE for integrated photonic waveguides and optical fibers. 5. Mikael Mazur from Chalmers University of Technology who talked about his experience of starting with Python and how to control an optical switch The event again resulted in great discussions about using Python for controlling lab equipment and for photonic design. Because of the great feedback that we received from these two events we decided that we will hold a hackathon again at OFC 2018. But a lot of feedback that we received also suggested that there is a need for a more permanent resource for using Python in the context of photonics, as aplace to get advice on how to get started, find events and instructions and howto’s or possibly link to relevant packages. We therefore started the https://python4photonics.org website, which we hope to develop into a hub for information on using python aimed photonics researchers. Currently the website contains information about the labautomation hackathons as well as information on how to install python and how to get started in learning python. We are planning to expand the website with more articles about using python in photonics, collections of links to photonics packages. There is also a blog which we aim to update regularly with new posts all around python and photonics. In conclusion we believe that Python is a great tool for the photonics researcher and that we will see its use grow significantly in the future. If you are a graduate student planning to automate your experiments (and you should do that if you are not already), I seriously recommend considering to use Python. Not only are you using a great tool that is fun to program with a large library of packages for all your needs, you are also developing skills that are extremely useful for a career after your studies, in case you decide not to stay in academia. If you are a more senior researcher, but are unhappy with your current automation tools, do look at Python. In particular if you are working in a non-academic lab moving away from matlab or labview could save you large amounts of money spend on licence fees.