The hottest Data Collection Substack posts right now

And their main takeaways
Category
Top Health Politics Topics

We could make our homes easier to decarbonise by using a crazy, sci-fi technology called a "database"

Odds and Ends of History • 737 implied HN points • 28 Feb 24
🌞 Climate & Environment Decarbonisation Database Data Collection
  1. Decarbonizing homes by replacing gas boilers with heat pumps is a known solution, but the challenge is logistical and costly for homeowners.
  2. Creating a central register of gas safety certificates could help target incentives for upgrading inefficient boilers, improve data collection, and hold landlords accountable.
  3. Adding gas safety certificate management to the Gas Safety Register's contract during the upcoming tender can facilitate the implementation of the central register at a minimal cost to the government.

Climate change is easy to prove if you screw the pooch badly enough

Tilting At Windmills • 334 implied HN points • 02 Feb 24
🌞 Climate & Environment Climate change Data Collection Scientific Integrity
  1. Climate change models have not accurately predicted outcomes despite drastic measures being proposed.
  2. Temperature readings used to support climate change claims may be inaccurate due to biases in monitoring stations.
  3. There is skepticism around the credibility of climate scientists and their data collection methods.

CDC Now Refusing New COVID Vaccine Adverse Event Reports in Its V-Safe Program

COVID Reason • 1985 implied HN points • 24 Aug 23
🏥 Health & Wellness Vaccine Safety Public Health Data Collection Medical Ethics
  1. CDC has stopped collecting adverse event reports for COVID vaccines on its V-safe website, directing users to the FDA's VAERS website instead.
  2. CDC not accepting new safety reports on potentially risky mRNA Covid-19 injections, raising concerns about monitoring safety of a new technology.
  3. Contrastingly, NHTSA continues to accept safety reports for a 30-year-old vehicle, highlighting the importance of ongoing safety data collection.

What can Colorado learn from Oregon about psilocybin?: 5 Questions for Healing Advocacy Fund’s Colorado director Tasia Poinsatte

The Microdose • 275 implied HN points • 05 Feb 24
🏥 Health Politics Psychedelics Advocacy Regulation Data Collection Mental health
  1. Colorado's psilocybin program is expected to be fully operational by early 2025.
  2. Oregon's program highlighted the importance of allowing licensed professionals to participate in psilocybin services.
  3. Colorado is considering a tiered licensing model and enhanced training requirements for safe facilitation in their psilocybin program.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Solving for Disease X

The Society of Problem Solvers • 176 implied HN points • 11 Feb 24
🏥 Health Politics Disease Prevention Public Health Measures Data Collection
  1. Using decentralized science and human swarm intelligence could help combat pandemics more effectively.
  2. Swarm intelligence involves group problem solving, where many individuals combine to form a more capable entity.
  3. A bottom-up approach with high-trust systems and decentralized problem-solving could lead to better solutions for combating diseases.

Better Descriptive Statistics for Low-Latency Profiling

Low Latency Trading Insights • 137 implied HN points • 06 Feb 24
🕹 Technology Statistics Data Collection Performance analysis Algorithm
  1. Better descriptive statistics are needed for low-latency profiling to accurately capture rare events and spikes.
  2. Descriptive statistics like mean, median, skewness, and kurtosis may be misleading in non-normally distributed data.
  3. Self-adjusting histograms with log-based ranges can provide more accurate data representation and efficient storage.

Training and Testing AI Systems

Rod’s Blog • 59 implied HN points • 28 Feb 24
🕹 Technology AI Data Collection
  1. Representative data is crucial for training AI systems to ensure they can handle various real-life scenarios and avoid biases.
  2. Challenges in collecting representative data include potential biases and incomplete datasets, which can impact the effectiveness of AI systems.
  3. Techniques like data augmentation can help address challenges in ensuring data representativeness by artificially diversifying and increasing the size of training datasets.

The World Bank African Countries Need

An Africanist Perspective • 613 implied HN points • 01 Mar 23
🌍 World Politics Development Data Collection International cooperation
  1. Low-income countries need the World Bank to focus on their real concerns to ensure program success.
  2. It's crucial for the World Bank to prioritize faster project implementation to avoid delays that disrupt policy planning and implementation in low-income countries.
  3. African countries should advocate for a World Bank that embraces big and transformative ideas, conducts better policy research, and improves data collection to accurately address the region's needs.

Applying the DevEx framework

Engineering Enablement • 14 implied HN points • 01 Mar 24
🕹 Technology Developer Productivity Frameworks Metrics Data Collection
  1. The DevEx framework focuses on the lived experiences of developers by measuring feedback loops, cognitive load, and flow state to enhance developer productivity.
  2. Teams interested in using metrics to improve developer productivity, such as platform engineering teams, engineering managers, and engineering executives, can benefit from implementing the DevEx framework.
  3. To successfully implement the DevEx framework, organizations should focus on getting feedback from developers, setting targets, driving impact through projects, running experiments, and then measuring progress to improve developer experience and productivity.

Why product analytics is completely different than BI

timo's substack • 294 implied HN points • 28 Feb 23
🕹 Technology Data Analytics Product Management Business Intelligence Data Collection Data Modeling
  1. Marketing analytics, BI, and product analytics have different requirements for source data and data handling.
  2. Product analytics involves more exploration and pattern-finding compared to marketing analytics and BI.
  3. Adopting product analytics requires a different approach, mindset, and tool compared to traditional analytics setups.

The Making of Whisper: An In-Depth Exploration of its Training Data and Process

Amgad’s Substack • 39 implied HN points • 22 Dec 23
🕹 Technology Data Collection Data Quality
  1. OpenAI's Whisper ASR model stands out for its accuracy, made possible by releasing both its architecture and checkpoints under an open-source license, setting a new standard of innovation in the field.
  2. The training of AI models can be divided into supervised and unsupervised approaches, each with its unique strengths and limitations, with significant implications for achieving high-quality results.
  3. Data curation is a critical aspect of model training, with OpenAI showcasing the importance of maintaining data integrity through a meticulous process of automated filtering, manual inspection, and guarding against data leakage.

After the Honeymoon: The Virtual Threats with Threads

Pen>Sword • 119 implied HN points • 12 Jul 23
🕹 Technology Social media Privacy Speech Censorship Data Collection
  1. Threads is a social media app launched by Meta that aims to fill the void left by Twitter's decline.
  2. Threads has raised concerns about privacy, speech, and censorship due to its data collection practices, restrictions on deleting accounts, and aversion to political content.
  3. The app's emphasis on 'kindness' and 'friendly spaces' is in contrast to worries about potential censorship and the impact on user freedom.

Three methods for measuring GenAI adoption and impact

Engineering Enablement • 4 implied HN points • 08 Mar 24
🕹 Technology Data Collection Artificial Intelligence Developer Productivity Surveys
  1. Telemetry metrics like pull requests per developer and code review time can give a high-level view of how GenAI tools are impacting developer output, but they may not provide a complete picture of tool utilization and benefits.
  2. Experience sampling, where developers are surveyed in real-time as they use GenAI tools, can offer valuable insights into specific time savings and tool usage, helping organizations understand the effectiveness of GenAI.
  3. Surveys are useful for measuring developer adoption, satisfaction, and self-reported productivity related to GenAI tools, providing a different perspective to complement telemetry metrics and experience sampling.

When Whisper 1.0 Gets It Wrong: An Inside Look at Speech-to-Text Failures

Dubverse Black • 117 implied HN points • 19 Apr 23
🕹 Technology AI Speech Recognition Data Collection
  1. OpenAI's Whisper model, while impressive, still has limitations and failures in speech-to-text accuracy.
  2. Whisper's challenges include repeating segments, mixing voice and non-voice activities, and inaccuracies in timestamps.
  3. The drawbacks of Whisper 1.0 present opportunities for learning, adaptation, and further development in enhancing speech-to-text technology.

The $800 Billion Secret Inside Your Car: Data

Fight to Repair • 59 implied HN points • 04 Aug 23
🕹 Technology Data Collection Repair IOT Digital Rights
  1. California is investigating how car companies collect data, emphasizing the importance of data transparency and ownership for vehicle owners
  2. Vehicle data is projected to be worth $800 billion by 2030, highlighting the lucrative nature of data collection from cars for companies
  3. Consumers often lack awareness of the data being collected from them, leading to potential privacy concerns and issues with car companies' practices

The Metaverse is a good business move[Finance Fridays]

Technology Made Simple • 179 implied HN points • 22 Oct 22
🕹 Technology AR/VR Social media Data Collection Engineering Skills Development
  1. The Metaverse is viewed as a beneficial business move, despite criticism from some sectors. It offers potential for immersive AR/VR experiences that could transform various industries.
  2. Critics raise concerns about the Metaverse's impact on mental health, utility versus costs, and accessibility to all. However, these challenges might not be as significant as initially perceived.
  3. Investing in the Metaverse could help Meta address its major challenges, create new revenue streams, and establish a unique position in the tech industry. Developing skills related to AR/VR and technology can potentially lead to opportunities in this evolving landscape.

Tracking/Measurement/Collection/Creation - what was the question again?

timo's substack • 78 implied HN points • 26 Mar 23
🕹 Technology Data Engineering Analytics Data Privacy Data Collection
  1. Finding a niche involves identifying what you enjoy and what is consistently needed in your projects.
  2. Tracking data is easily understood, but may have a negative reputation due to its association with web tracking practices.
  3. Measurement is a broader term than tracking, and data collection is often overlooked in the data engineering process.

We've Entered the Era of Hyper-Personalization

Embracing Enigmas • 39 implied HN points • 31 May 23
🕹 Technology Personalization AI Data Collection Algorithms Content creation
  1. We are entering an era of hyper-personalization where content is tailored to specific individuals beyond just what they might like.
  2. The progression of personalization stages includes one-size-fits-all, segmentation, behavioral personalization, predictive personalization, and now hyper-personalization.
  3. The main components needed for hyper-personalization are data about the individual, algorithms for content selection, content creation, and a trust layer for quality control.

🚨 Datafication of Work: Tools, Ownership and AI

Work3 - The Future of Work • 39 implied HN points • 27 Feb 23
🕹 Technology Data Collection Blockchain Ethical considerations
  1. Companies are collecting various types of data about employees, like productivity metrics and performance reviews, which is shaping the future of work.
  2. There are benefits and ethical concerns with the datafication of work, as it raises questions about ownership and use of employee data.
  3. The use of AI in hiring and performance management is increasing, prompting the need for transparency, diversity in development teams, and ethical guidelines.

A user researcher's guide to secondary research

UX Psychology • 99 implied HN points • 04 Feb 22
🕹 Technology Research Academic Data Collection
  1. Secondary research involves using existing data to answer research questions rather than gathering new data directly. It helps deepen understanding of the problem space and can save time by guiding primary research.
  2. Conducting secondary research starts by defining the research question, identifying potential sources, evaluating source reliability, conducting the search, and creating a report or summary. This process is crucial for gathering reliable information.
  3. Using academic principles like literature review can enhance the quality of secondary research in UX projects, helping to shape research questions and hypotheses based on existing knowledge.

Lessons from Accelerating Knowledge

Embracing Enigmas • 19 implied HN points • 02 May 23
🕹 Technology Machine Learning Data Collection Knowledge sharing
  1. Machine learning progresses quickly due to factors like the leaderboard effect, ease of experimentation, and decreased cost of computation.
  2. Researchers and practitioners in machine learning benefit from sharing knowledge and ideas, leading to rapid improvements in the field.
  3. Machine learning's broad applications across various industries contribute to its growth, attracting investment and fostering cross-pollination of ideas.

First Mile Observability and the Rise of Observability Pipelines

Bit by Bit • 8 implied HN points • 14 Aug 23
🕹 Technology Observability Data Collection Open Source
  1. Observability extends beyond just backend systems to include the 'first mile' of data collection and processing.
  2. First-mile observability involves components like receivers, processors, and exporters to create observability pipelines.
  3. Various open-source and commercial solutions exist for implementing first-mile observability pipelines, with options like Vector, Fluent Bit, OTEL Collector, Cribl, Calyptia, Datadog, and Mezmo.

A Tourist's Guide to Datasets

Cybernetic Forests • 59 implied HN points • 04 Jul 21
🕹 Technology Data Collection Machine Learning Data interpretation Data Sharing Digital Art
  1. Machines understand models of reality through data, influenced by what is deemed significant, leading to gaps and potential misinterpretations.
  2. Datasets are contextual and not universally applicable, emphasizing the importance of clear documentation and awareness of data limitations.
  3. Creating a 'Tourist's Guide to Datasets' with annotations and personal insights can enhance understanding and avoid misuse when data is reused for different purposes.

10/36 - User Research 101: Navigating the Discovery Process

Product Mindset's Newsletter • 9 implied HN points • 19 Mar 23
🕹 Technology User Research Product Management Data Collection
  1. User research helps understand user behaviors and needs through various methodologies to improve product usability.
  2. Benefits of user research include cost reduction, increased user satisfaction, and gaining a competitive advantage.
  3. User research methods include qualitative and quantitative approaches, attitudinal and behavioral studies, and a mix for a comprehensive view.

Improving data quality in online studies

UX Psychology • 19 implied HN points • 23 Nov 21
🔬 Science Data Quality Data Collection Quality Assurance
  1. In online studies, factors like distractions, poor equipment, and cheating can impact data quality.
  2. Engagement levels, accuracy, outliers, and speed of responses are key indicators to assess data quality in online studies.
  3. Strategies like consistency measures, attention checks, bot detection, and serious response checks can help improve data quality in online studies.

The Greatest Trick Big Brother Ever Pulled

Global Community Weekly (GloCom) • 0 implied HN points • 11 Feb 24
🕹 Technology Facial Recognition Privacy Concerns Law enforcement Data Collection
  1. The surveillance state is gradually emerging in small towns through various surveillance gadgets like facial recognition, gunshot detection devices, and automatic license plate readers, posing privacy threats.
  2. Facial recognition technology has raised concerns due to its use for petty purposes, leading to harassment and wrongful arrests, prompting efforts to ban its government use.
  3. Surveillance gadgets like automatic license plate readers are being promoted as non-threatening and old-fashioned, but concerns exist about privacy violations and their effectiveness in preventing crimes.

Study 1b Pre-registration

From AI to ZI • 0 implied HN points • 17 Apr 23
🔬 Science Research Analysis Data Collection Transparency
  1. Study 1b aims to rerun Study 1a with a different prompting method to potentially increase the rate of factually incorrect answers
  2. The study will test hypotheses related to the accuracy of large language models under new prompting formats
  3. The data will be analyzed using multiple-regression analysis to determine the effects of different variables on the model's accuracy

Running out of things to say

Joshua Gans' Newsletter • 0 implied HN points • 19 Mar 21
🏥 Health & Wellness COVID-19 Economics Data Collection Testing Vaccines
  1. The author of the newsletter is taking a break due to running out of things to say after consistent writing for a year, but shares interesting articles from other sources.
  2. The shared articles cover various topics related to Covid-19 such as the importance of data, testing failures, new testing methods like rapid screens, and the need for continued testing even with vaccines available.
  3. The post also links to a new book called 'Economics in One Virus' by Ryan Borne that takes an economic perspective on situations arising from the pandemic.

«Правильная» мобилизация: как изменится система воинского учета в России

Faridaily • 0 implied HN points • 18 Feb 23
🌍 World Politics Military Data Collection Borders Security International relations
  1. Russian authorities are creating a comprehensive database of military conscripts to facilitate faster mobilization if needed.
  2. Various government agencies will share citizen data to populate the database, including information on residence, health, employment, and more.
  3. The new system aims to prevent mistakes and improve efficiency during mobilization, making it harder to evade military service.

Báo Nói Application: A Guide to Building High-Quality Text-to-Speech Datasets

CodeLink’s Substack • 0 implied HN points • 28 Jun 23
🕹 Technology AI Data Collection Open Source Text-to-Speech
  1. High-quality data is essential for training accurate and natural-sounding text-to-speech AI models.
  2. Cutting-edge tools like annotation software and ASR services are pivotal for efficient data collection in developing text-to-speech AI models.
  3. Collaboration and data sharing drive innovation in the AI community, enhancing the representation of diverse perspectives and voices in AI-generated speech.

Check 'em Privacy Policy

Jacob’s Tech Tavern • 0 implied HN points • 13 Feb 24
🕹 Technology Privacy Policy Data Collection Security
  1. The app Check 'em doesn't collect any data and doesn't even use the internet, ensuring user privacy.
  2. Users of Check 'em are not required to provide any personal information or create an account, emphasizing user anonymity.
  3. The app ensures high security by storing data securely on the iOS keychain and following best practices in generating 2FA codes.

The Changing Ad Landscape (and What To Do About It) | Youssef Ahres (Flagship, Instagram)

Steelhead • 0 implied HN points • 31 Jan 24
💼 Business Advertising Customer Engagement Brand building Data Collection Revenue growth
  1. Advertising serves to match supply and demand, with the value shifting to those who can effectively manage this in a world of abundant supply.
  2. Meta and Google have thrived in digital advertising by being widely used and investing in technology for targeted ad delivery.
  3. In the face of changing privacy concerns, companies should focus on leveraging first-party data, mastering customer engagement, exploring new advertising channels, and building strong brands to thrive.