Can data visualization help us find our true selves and make the world a better place?

We are on the cusp of a data boom. We already generate more data every day than at any other time in history due to the pervasiveness of mobile technology. However, the increased adoption of wearables and smart IoT devices in our homes, offices and public spaces, coupled with advances in machine learning will make those already impressive data volumes pale in comparison to what is to come.

We are on the cusp of a IoT data boom
Share on X

Established out of Stanford in 2003, Tableau Software has been in the data visualisation space for nearly 15 years, and has spearheaded the trend that has thankfully seen us move away from Excel spreadsheets and those good old pie charts. Sure, they’re still around, but cutting-edge technology such as Tableau’s now works behind the scenes to turn data into actionable answers and insights in a much more intuitive and productive way that doesn’t require a knack for numbers or special training by the user. Data, Francois Ajenstat, Tableau Chief Product Officer believes, should be made as approachable as possible, and to many more people.

Tableau’s public platform enables over 300,000 people to share their data stories for free
Share on X

It is difficult to grasp the sheer quantity of data individuals will be generating and consuming in the near future but according to Cisco, global IP traffic will increase nearly threefold over the next 5 years, and will have increased 127-fold from 2005 to 2021 and overall, IP traffic will grow at a Compound Annual Growth Rate (CAGR) of 24 percent from 2016 to 2021. It is clear this growth will continue and as our data piles up in cyberspace, both humans and technology will have to advance to engage with it.

Visualization tools will help humans make sense of Big Data
Share on X

Soon over 40% of that data will be generated by IoT connected devices, and this shift is already revolutionizing many industries and creating completely new ones. Most of this data will come from sensors, geolocation, temperature, proximity, motion, humidity, pedometers and many more that will increasingly record information about every aspect of our lifestyle and habits. As the IoT trend accelerates and continues to connect more and more devices Internet, those sensors will create high volumes of data in a short timeframe so new systems need to be able to quickly compile data from sensors that might not be located in a single location, city, country or even continent.

People can tap into IoT data streams to generate valuable insights
Share on X

To handle this unimaginable volume of data, the backend infrastructure that shares and stores it will have to become faster and more agile, while at the same time we will have increasingly sophisticated systems that process and translate these billions of rows of information into useful insights. Visualization, in other words, will be crucial in helping humans make sense of it all. And that is a huge emerging market that is spanning many different technologies. At Microsoft Build, for example Tech Trends experienced how Data Visualization is one of the most compelling applications for Mixed Reality.

“Data is the electricity of the 21st century, it powers everything, and increasingly, with IoT, everything is generating data, your phone is generating data, your home is becoming smart, and your car is getting smart, stores and products are generating data, so how do you bring all that data to life and make use of it?” asks Tableau’s Chief Product Officer Francois Ajenstat.

Tableau’s business model works on providing those valuable visualization tools and services for businesses, but they’re also passionate about democratizing access and advancing “data empowerment” on an individual level. Its Public Platform – which Francois describes as a “YouTube of Data” – is nothing short of fascinating, showing that citizens and ordinary people want to engage with data and use it to drive productive actions and broader conversations around topics as varied as Election Turnout, Global Warming, or a shared loved for The Beatles. As of May 2017 it has enabled over 300,000 users to share their data stories, sharing their own topical data analysis for free with a global community.

We generate more data every day than at any other time in history
Share on X

Many of us already routinely monitor how many steps we take each hour, how long we sleep, how much we eat of what… In a world where our every heartbeat and brainwave will be recorded, we won’t need psychiatrists to delve into the depths of our quantified selves, we will need good data vis. Millions of people, for example, use devices such as the Fitbit to monitor their personal data, which is visualized and analysed through a now-familiar style dashboard. As Tableau’s Senior Technical Evangelist Andy Cotgreave explained in a recent article, our relationship with such interfaces needs to reflect the new reality where our data is constantly evolving. A Dashboard, therefore, should evolve with our data, and be seen not as a finishing line, but as a starting point, empowering users to better know their quantified selves.

Data visualization is a compelling use case for Mixed Reality
Share on X

IoT applications usually come with pre-built data visualizations which are quite boring to look at and answer only a few standard questions that cannot be customized. Yet the real value in IoT data only emerges when users can get their hands dirty playing with their data and exploring its limits. “You want the user to be able to say, tell me more, I have a new question, and then another question.” Francois enthused, “Tableau wants to give people the most clear visualisation so they can then engage with it in and understand the data. Easily seeing patterns, trends, and making meaningful comparisons and connections.”

Data lets people understand the world around them and have an impact, concludes Ajenstat. “Being able to access your data or data about the world around you is empowering, as people are better able to engage with the world around them and change it for the better,” he says. “We are investing in AI, algorithms and machine learning to help bring data to life. We believe AI can help augment the human, and make the human smarter.”

For companies looking to get into Immersive technologies such as VR/AR/MR/XR our Virtual Reality Consultancy services offer guidance and support on how best to incorporate these into your brand strategy.

Tom Atkinson is a Digital Producer & Photographer at R3Digital and Reviews & Dept. Editor at Tech Trends. Connect on LinkedIn and follow him on Twitter @R3Digital

The post Analysing your Quantified Self appeared first on Tech Trends.

Making the huge volume of data out there useful to humans is about contextualizing it, but if Facebook has taught us anything is that this must be done responsibly.

Back in 2012 Forbes Magazine declared that data was “the new oil,” and nobody these days disputes the fact that there is a ridiculously large amount of useful data available out there. The use of HTTP as an access method and semantic web languages as interchange formats have turned the Web into the largest decentralised database the world has ever seen. The problem we face, however, is that there are major issues around reliability, accessibility and socialisation of that data that stop it from being as universally useful as it could be.

There are challenges in extracting scientific data from PDFs
Share on X

It was Tim Berners-Lee who once said that the next evolution of the World Wide Web – or Web 3.0 if you prefer – would be about the “Giant Global Graph”. What he was talking about was Big Data, but in a social dynamic context which people can easily access and take advantage of. In the words of Gerhard Weikum, Research Director at the Max Planck Institute for Computer Science, nearly all experimental “Big Data” is “utterly boring,” with evaluations ending up in “completely synthetic data with a synthetic workload that has been beaten to death for the last twenty years”. To make this data “interesting”, what he proposes is to bring Big Data and Open Data together, creating Linked Open Data.

Major administrative authorities already publish their statistical data in a Linked Data aware format, but the actual value of these datasets is not unleashed or fully exploited, because data needs context to be of value, and “socialising” is what provides such context. One example of this is the Digital Agenda EU Portal, which has a huge number of datasets on important European indicators, but does not allow people to share their findings or to discuss its interpretations. This means that the context, which gives the data most of its meaning, is simply missing.

As a researcher, you tend to spend most of your time trying to make sense of datasets
Share on X

That is the problem that a group of EU-funded researchers are trying to tackle, together with industry partners such as London-based research collaboration platform Mendeley. They launched an open beta version of 42-data, a portal that aims- as the name will suggest to any fans of “Hitchhiker’s Guide to the Galaxy” – to provide answers to the universe and everything by socialising statistical data. This is the main output of theCODE project, which has a remit of facilitating “community-driven insight generation” by lifting non-semantic web data silos in an ecosystem around Linked Open Data, bootstrapped by micropayments and trust mechanisms. Their goal is to essentially create a “flea market for research data” by combining crowd sourced workflows with offline statistical data. This would create a Linked Open Data cloud capable of generating customised datasets to backup and answer all manner of research questions.

Scientific articles are obviously the perfect fodder for this cloud database, but they come with a major problem attached: most papers are in PDF format, which means that it’s difficult (not to say impossible) to extract the primary research data contained in tables and figures. The CODE project, which Mendeley participates in, addresses this by reverse-engineering the paper to extract this information in a format that can then be easily processed and analysed.

“We have a long-standing partnership with Mendeley which started with the TEAM-Beam project, and was then extended to CODE,” says Professor Michael Granitzer, from the University of Passau, Germany, the academic partners responsible for the 42-data portal. “The vision with CODE is basically to make the daily lives of researchers a bit easier, and Mendeley is the perfect partner for that, because it already offers so many tools like the group collaboration and the open API. In the scope of the CODE project, we developed and deployed lots of tools to analyse research publications. Most of that analysis consists of information from inside the paper itself, the primary statistical research data such as tables. We enrich this analysis with linked Open Data to generate meaningful insights and broaden a researcher’s view in a user-friendly way, with sophisticated visualisations that can generate interactive charts and other assets for their research,” he explains.

As a researcher, you tend to spend most of your time actually making (or trying to make) sense out of datasets, and this process means that you have less time to come up with interesting insights and advance research in your field. Take, for example, a researcher preparing to write up their paper: Why is the proposed approach better? The hypothesis they’re putting forward must be backed up by meaningful data, so they are faced with the task of extracting and aggregating statistical primary research data that is stored in tables within various research papers, and then combining, comparing and contrasting this with their own evaluation data. Without integrating these workflows, you’d need a plethora of tools, specially since copying and pasting from PDFs does not work for this type of data. Within 42-data, however, Mendeley hosts and pre-processes those papers, using the Know-Center services to extract the information in a format that is easily processed and manipulable.

The platform itself collects that table-based data and the University of Passau uses Mendeley’s API to merge all those single results accordingly, creating a “data cube” of merged and linked data. This data cube presents the researcher with an integrated view of all those disparate data sources. “But that’s not even the end of it, as a data cube can then be enriched with Open Data to offer up even more insights,” concludes Granitzer. The analysis and discovery is thus not limited to the initial dataset, as the platform offers virtually endless possibilities for customised mixing and matching within what 42-data calls “Data Cubes” to address specific research questions and needs. Individual cubes can be interconnected and aggregated using a graphical interface, which guides the user through and warns of any integrity constraint violations, and how these can be solved, by modifying its structure.

This uncovers some exciting possibilities for accelerating scientific discovery; if some of the sensemaking legwork was automated by such portals, we could see the emergence of a virtual meeting place for people interested in getting insights from such Open Data sets, similarly to how Mendeley users interact in groups based around their research interests. “It is a well-known fact that discoveries in academia come out of intense communication processes, and that is what we’re looking to support,” says Florian Stegmaier, Senior Researcher at the University of Passau. “In addition, the social/crowdsourcing aspect of the platform means that we’re going way beyond the text-based model of asking questions, broadening the scope of discussions to include virtually everything. You could assess the suitability of your research ideas based on existing data, ask for statistics to be included in a paper, or simply discuss a range of published papers to get an in-depth view of the subject,” he enthuses.

But analysing, integrating and sharing data comes with associated costs, as does running such a portal. Beyond the initial EU Seventh Framework Programme grant, how does 42-data actually propose to fund itself? “It’s crucial to establish a value chain for data that creates a positive benefit-to-cost ratio, and we are doing that through two main mechanisms: Reputation and Donations,” Michael Granitzer explains. Reputation is certainly the core motivation driver in the crowdsourcing ecosystem, as we’ve seen withStackExchange and Github, amongst many other high-profile examples. They set out to provide a similar proposition, where users contribute to open-source data projects, analysing data sets and creating interesting insights.

In order for this reputation model to work within the Web of Data you need to establish provenance. This means there is a solid chain of data, which tells you the origin or source of every individual piece of information within that chain. That includes records about which individuals were involved in creating, changing or extracting the data at any given point in time. If a particular person generates a data cube with their query, their ID is stored in that cube to guarantee this reproducible mapping (in the case of data extracted from a paper hosted on Mendeley, the metadata referring to the author name, abstract, publication date, academic status, discipline, research interests, etc. is automatically extracted and linked to the cube). The plan as the platform develops is to triangulate this information with community ratings and recommendation algorithms to produce a “user trust score” that will further feed the reputational ecosystem.

What we see with data today is a similar situation to what we had in the era prior to Web 2.0
Share on X

Donations also provide monetary incentives, in the community-driven financing model that Wikipedia pioneered. A “revenue chain” is created by allowing people to donate to users, questions, answers or resources that they find particularly helpful. The idea is to explore the long tail of micro-payments by keeping it flexible. You can target your donation to a specific user, or if it’s a collaborative effort, this can be sent to multiple targets, with user’s trust and reputation scores on the site also influencing how well they do out of those transactions, which is hoped will foster a stronger and more cooperative community. “The complete ecosystem is driven by trust and reputation mechanisms. The higher the trust is, the more likely one will donate for something,” says Granitzer.

What we see with data today is a similar situation to what we had in the era prior to Web 2.0, where there was a lot of content around, but socialisation over that content was not enabled. Just as we’ve seen with the social media boom of recent years, however, there is now an opportunity and appetite for creating communities of interest around the socialisation of data. Through exploring Linked Open Data, users should be empowered to aggregate and integrate interesting data, quickly tailoring it to their specific research needs. That is, however, just the first step, as this increased socialisation could make these datasets accessible to non-scientists as well. The growing momentum of the Citizen Science movement goes to show the enormous potential of opening up science in this way, and the possibilities that this opens up are truly amazing.

Exploring Linked Open Data, users could aggregate interesting data and tailor it to their specific research needs
Share on X

Originally published in the Huffington Post

Alice Bonasio is a VR Consultant and Tech Trends’ Editor in Chief. She also regularly writes for Fast Company, Ars Technica, Quartz, Wired and others. Connect with her on LinkedIn and follow @alicebonasio and @techtrends_tech on Twitter.

The post Making Big Data More Social appeared first on Tech Trends.