What is data science? The complete guide on the subject

 All the new online platforms like Youtube, Netflix, Snapchat, Facebook, and Instagram generate huge masses of data. These are the so-called data-driven companies, data-oriented companies that use data science to make decisions.

In this article, we will explain what data science is, where it comes from and what its main applications are. In addition, we will also address its main subdivisions and what are the possible professions to follow in this area.

The data revolution

Technological innovation generates advances that transform the world around us and empower us as individuals. One of the important effects of the digital transformation was the democratization of knowledge, today virtually free.

However, taking advantage of the full potential of this digital transformation is only possible if we exploit the capacity of the data generated by these innovations. In addition to the abundance of available data, what drives the revolution in the data industry are technologies that change the way we collect, store, analyze and transform information.

Throughout human history, the milestones of our civilization have been characterized by advances in our ability to observe and collect data. Our distant ancestors developed practical tools and methods for measuring distance, weight, volume, temperature, time, and location.

All this experience was fundamental to their transformation from hunter-gatherers into farmers and, later, into inhabitants of cities with more complex organizations and divisions of labor.

Throughout modern history, even small amounts of data have provided us with important insights in finding solutions to some of our biggest challenges. Recording information on the stone, papyrus, printed books, and later computers has been one of the main drivers of human progress.

Data acceleration

In this century, we are experiencing a rapid acceleration of this whole process. As data becomes more abundant and its storage and processing costs decrease, data scientists benefit from technology to unlock valuable insights.

With advanced data analysis tools, data science professionals can make predictions that solve big problems and improve our everyday lives. Today, transportation technologies, for example, are being disrupted by new applications based on large volumes of data, such as Uber and Waze.

The beginning of data science

Some factors culminated in the existence of data science. The main one is the increase in unstructured data available, from the digitization of information. This large volume of unstructured data is also known as Big Data.

The second important factor was the advancement in cloud processing capacity, through horizontal processing with clustersWithout this increase in processing power, data science would certainly not exist. This is because traditional vertical processing is expensive and inefficient for large amounts of data.

This problem was mainly solved from the specialization of computing power made available by cloud computing providers such as Amazon (AWS), Google (GCP), and Microsoft (Azure). With the possibility of leasing hardware on demand and its redistribution to achieve maximum efficiency, many projects have become feasible with cloud computing.

What is the difference between data and information?

Conceptually, data are quantifiable contents that have no value, is considered a basic unit of value. Information is the result of data processing, that is, the interpretation of data or its meaning.

In the diagram below, it is possible to understand how the flow of transformation between data and information works:

Input (data) → Processing (data analysis) → Output (information)

In today's world, the greater the amount of data, the greater the amount of information available and, consequently, the greater the knowledge acquired by humanity. All this generates the need for people to interpret data and, from that, create predictions. This professional is the data scientist.

In the past, most data was unprocessed, that is, it was not transformed into information. Today, with the processing power in the cloud, companies are looking to transform data into information to interpret it and generate important insights for their business.

Insight concept

Usually, businesses have problems that need solutions. In turn, solutions require decisions based on data.

Insight is the solution or conclusion about something. If I have a problem I can conclude something about that problem, which would be in sight. From a business point of view, every decision process should be based on data, hence the importance of insights.

With the concepts of data, information and insight solidified we can start to try to understand data science.

What is data science?

Today, the data science profession is the fastest growing in the world. Much of this is caused by the need that companies have to treat unstructured data and transform it into useful information.

According to experts, it is estimated that around 90% of the data stored on the web has been generated in the last 2 years aloneIn addition, a maximum of 20% of this data is structured in rows and columns to be analyzed by traditional tools. A video uploaded to Youtube, for example, is considered unstructured data, as it is composed of images and audio. That is, it does not have the information organized into categories (labeled).

Data science is the collection of data from various sources to analyze and support decision-making, in a predictive way, in large quantities and generating insights.

It is important to remember that prediction does not guarantee the future, it is just a tool to improve the decision process. That is, planning is not the certainty, as it is not immune to failure.
Data science, as it is known in Portuguese, is the process that extracts data from different sources, at different speeds, processing large amounts (big data) and generating value. In no way can it be understood as a tool, but as a set of methods, just like big data and business intelligence.

Pillars of Data Science

Among the main pillars of data science are mathematics, statistics, business, data mining and visualization, programming, and computing. Although this area is essentially multidisciplinary, statistics and mathematics are the basis of data science and the differential of previous methods, since it is through them that data analysis models for future prediction, also known as algorithms, are built.

What is the difference between Data Science and BI?

There is some confusion about the difference between data science and business intelligence (BI). This is not surprising, since the two disciplines are very similar and use large amounts of data as input. However, despite working towards the same goal, their approaches, technologies, and functions differ in many ways.

The purpose of business intelligence is to convert raw data into business insights so business leaders can make decisions. The BI professional, the business analyst, uses tools to create management support products, such as dashboards and reports.

Data science, on the other hand, employs the scientific method for exploring data, forming hypotheses and testing hypotheses, through simulation and statistical modeling. Within data science, machine learning is still used as a tool to automate the transformation of data into information.

The main difference between the two is that business intelligence deals with data from the past while data science will deal with the future, based on predictive analytics. Sometimes, the BI professional can even make some predictions about the future, but they are based on extrapolations from the past, that is, they do not use a scientific basis.

And what is machine learning anyway?

Machine learning is a subfield of artificial intelligence where a machine is programmed to learn from collected data. This learning can be supervised or unsupervised.

An example of machine learning is image recognition which allows artificial intelligence to achieve satisfactory probabilities and a high degree of accuracy for categorizing photographs. A good example of this is supervised learning by Google users by filling out a captcha that helps artificial intelligence identify patterns and differentiate objects in images.

Data science applications

Data science has many practical applications. Some of them are product recommendation in online retail, voice recognition (deep learning), treatment of diseases based on data correlations, and facial recognition.

Today, many technology vendors are investing heavily in deep learning technologies for speech recognition. Cortana (Microsoft), Siri (Apple), and Alexa (Amazon) are some examples of conversational technologies, which allow the user to interact with artificial intelligence through voice commands. This technology reveals in a very comprehensive way how the transformation between unstructured data (voice) into useful information (computational commands) works.

Data Science Careers

The data science career is one of the most promising right now. Although there are numerous possibilities of professions to follow, there are 3 macro profiles that aggregate all the current possibilities of the profession.

1. Data scientist

Professional with a strong background in exact sciences, such as computer science, mathematics and statistics. data scientist is able to analyze large amounts of data and come to conclusions (insights) or generate predictions. It is certainly a more complete profile, which mixes business and exact knowledge.

2. Data Engineer

The Data Science area also needs a professional with a technological and infrastructure profile. Due to the large amount of data that this professional will work with, it is necessary to administer clusters for parallel processing of data, whether structured or unstructured.

The data engineer must be able to prepare the data, creating data lakes and data warehouses for data scientists to consume.

3. Business Analyst

As in the BI area, the Data Science area also needs professionals with a business profile, who are able to understand the heart of the company, and suggest new practices or businesses in order to generate more value.

The Future of Data Science

Data science, through its predictions, informs us in advance whether we should leave the house with an umbrella, which is the best way to get to work and which movie we are most likely to like based on our previous preferences. As data moves from being a scarce resource to increasingly abundant, data becomes an essential source of social and economic benefits.

With the cost of storing and processing data falling and with the increase in the number of sensors that capture more and more information, the amount of data available will be increasing, as will the possibilities of using this data. We live surrounded by opportunities generated by data, which can give us answers to some of the biggest challenges in the world, such as greater efficiency of health resources or the restructuring of transport systems.

It is up to professionals in this new field of science to create models to enhance productivity in all areas. There is no restriction in any area for the work of data scientists, which is a great opportunity to make the human effort more and more efficient.

Post a Comment

Previous Post Next Post