In the past, I very frequently had occasion to work in multidisciplinary groups with professionals, researchers, technicians, medical and health personnel, biologists, statisticians and engineers. I always thought that one of the most interesting aspects of these projects was the opportunity to learn to further recognize one’s limits and live with those of others.
But, in practical terms, how is it possible to work in a multidisciplinary group in a context like that of Analytics, a sector in which professionalism is often measured according to the level of specialization and in which even the user (unfortunately) is sometimes tempted to accept the results “sight unseen”?
Many heads
In the world of Analytics, ever different spirits come face to face: that of the expert in business or domain, with a technical or commercial background (in multidisciplinary projects they may not be a technician at all, but a specialist in the field, like a doctor); that of the developer or system engineer, often with a computer science background; that of the analyst, generally specialized in “quantitative” disciplines (scientific, economic, etc.). The differences translate into different points of view and, often into the use of different technologies, which must necessarily be integrated for a project to be successful. Faced with this difficulty, the constant temptation is to limit yourself to translating your professionalism into documentation to be entrusted to your partners and to relegate, or grab, the responsibility for “synthesizing” results according to your unique point of view. Although a phase of synthesis is obviously necessary, the most successful and interesting projects you remember having participated in were those in which complementary professional skills were managed, rather than synthesized, by one or few members. To be able to do so, there was a need for the right tools, a common and immediate language, an approach based on agile methods and mutual trust.
It should be clarified that the proposed vision is a necessary simplification of reality. Here is another entertaining and interesting point of view.
Modularity and agile approaches
Adapting a phrase from an interesting book, Agile Data Science 2.0: Building Full-Stack Data Analytics Applications with Spark “information has context, and when that context is interactive, the contents are not predictable”. Like saying: the success of an Analytics project depends largely on the quality of the data (that is, of the information) available. This is explained by the fact that Analytics projects are generally data-driven (literally: guided by data): the outcome is not the software itself, but rather the information extracted from this software, which changes as the data change. It is therefore essential for each member of the group to be able to adapt to change, as sudden as it is frequent. An effective solution for facing up to change is to create (when possible) relatively small working groups, made up of people with extremely varied skills, promoting frequent exchanges of opinions “among peers” (or almost). Exchanges should be supported by tools that allow systematic access to data and to even partial processing. This approach allows you to work iteratively and efficiently, optimizing the time necessary for communication, but requires each member of the group to be willing to study and constantly expand their professionalism with new skills.
Collaborative tools
Projects involving the development of Machine Learning systems generally call for frequent validations, rapid sharing of results and frequent consultations among all the spirits of the group, or with the client. Therefore, the choice and use of sharing tools which allow access to documents, presentations, codes and the viewing of data in a collaborative and, possibly, interactive way, according to different levels of complexity, is crucial. Most data visualization systems today support working in the cloud. Starting from initial canvases, be they documents, presentations, dashboards or code, and starting from compliance with a few essential requisites, everyone can add their own contribution during the project, based on their role and degree of involvement. Clearly, this presupposes effective communication among members of the group and careful direction.
Common language
I do not believe that the importance attributed to communication among professionals is ever exaggerated, even more so if from different cultural backgrounds. The use of technical terms, while referring to shared concepts, is often so different as to generate delays, major misunderstandings and endless discussions. Sometimes it is a matter of facing the challenge of translating concepts by simplifying them, but without trivializing them to the point of being superficial and inconclusive. Essentiality is also a talent that I believe should be cultivated. Not without reason, many postgraduate courses, including technical courses, now offer training programs in the field of soft skills (presentations, communication, etc.). In fact, it can be a challenge for an analyst to translate their results in a synthetic way so that they can be validated by a domain user; for a computer scientist, it is not always easy to give value to a technological solution, even though defining its perimeter, which arises from the compromise between project requirements and budget. Poor communication undermines the effectiveness of project results.
Tools of “synthesis” and osmosis of skills
In the field of IT, one of the priorities is often to “release” algorithmic solutions, which have the characteristic of being modular, often based on Machine Learning systems and intended to be integrated into broader platforms. From a strictly technical point of view, one of the most effective tools which enables the integration of analytical components is that of APIs or microservices which, regardless of the application solution or language used, allow you to query models and make forecasts. Environment virtualization systems (such as Docker) combined with Continuous Integration tools (such as Jenkins), make it possible to develop microservices in a scalable and modular fashion. Analysts can concentrate on producing content, leaving the complex task of drawing the details of integration to the more technical profiles (oriented towards DevOps type professionalism) and the possibility of consulting data regardless (or almost) of the platform used to domain experts. Rather than producing only documentation to entrust to the implementation capabilities of colleagues, each member of the group thus becomes directly responsible for one component of the project, which is developed and integrated together with colleagues.
For those wishing to explore further, see an entertaining article born from a seminar held at Strata+Hadoop World, London, 2016.
Trust
Last on the list, but probably first in terms of relevance, trust is earned, but sometimes must also be agreed, above all in first experiences as a group. As is natural, a climate of trust allows the establishment of a fertile human environment for encouraging exchanges of ideas and skills to focus on common objectives. This climate is strengthened by promoting opportunities for contact among people and sharing spaces and objects, but also by promoting opportunities for confrontation within the group itself. In my experience, the confrontation that arises from serene sharing, especially in multidisciplinary groups, always leads to a growth of one’s views and knowledge which, I believe, can be considered one of the highest aspirations for anyone who dedicates care and passion to their work.
Michele Gabusi
