6 min read
Scroll to the bottom for summary.
Today’s trend could be tomorrow’s turd.
Resources
To stay up-to-date (with tech, stocks, politics, high profile banters), we try and consume information from many sources. A lot of us spend our precious time and effort in finding those relevant sources. To save you (and my colleagues) that time and exertion, I have done the due diligence of human web-scraping to compile a list of useful resources that range from data engineering to data science, ingestion to visualization, lakes to marts and so much more.
- Datanami
Datanami is a comprehensive portal, delivering in-depth analysis and up-to-the-minute updates on emerging trends and advancements in big data. It features interviews with industry leaders, shares success and struggle stories from organizations, offers a weekly newsletter, hosts an annual conference for data professionals from around the world and also, has a user forum for discussions.
Fun fact - "Datanami" is a portmanteau of the words "data" and "nami", where nami means "wave" in Japanese. - Towards Data Science
Towards Data Science is a Medium publication that publishes articles on data science, machine learning, and artificial intelligence from thousands of independent authors. It offers content, including tutorials, case studies, and interviews with industry experts. The audience is diverse, including all skill levels, from beginner to expert. The TDS team vets every post, ensuring it’s relevant and valuable to the readers. They have a library of code and data sets and also have a podcast that interviews thought leaders and data experts. - Seattle Data Guy
SeattleDataGuy is a Substack newsletter by Ben Rogojan, a data infrastructure consultant and a data engineer. Rogojan has almost a decade of experience in the field of data and is passionate about sharing his knowledge and expertise with others. He not only focuses on the the tech aspects of data, but also the people and processes involved in it. I think he does a great job of explaining complex concepts in a clear and concise way. - Dataversity
Dataversity is a website that provides educational resources on the uses and management of data. The website offers a variety of content, including online courses, white papers, news and events. They also have a weekly newsletter and a forum where users can discuss data management and analytics. Dataversity is for anyone who wants to learn more about data management and analytics. Whether you're a business leader, a technologist, or just someone who's curious about data, Dataversity has something to offer you.
For 26 years they have hosted the globally recognized Enterprise Data World Conference, a comprehensive educational conference on Data Management in the world. - Database Trends and Applications
For some of us who still like consuming information through print, DBTA also offers a magazine subscription. Both, the magazine and the website cover technologies and processes involved in every aspect of the creation, management, application and storage of data. The content is original, factual and valuable. They have a variety of newsletters tailored to specific audience, covering topics in Data Center, MultiValue, Cloud Strategies and many more. - The Data Administration Newsletter
TDAN.com is a digital publication centered on providing knowledge about data management strategies, best practices and technological solutions. This platform contains many insightful content, contributed by both seasoned industry leaders and individuals actively involved in various aspects of data management. It has particularly fostered a community of passionate data professionals who engage in insightful discussions and knowledge sharing, contributing to the advancement of the data management field. - Transforming Data with Intelligence
TDWI is a website and organization that provides resources and training for data professionals. It also has a job board and a community forum where data professionals can connect with each other. Not only does it have a wide range of content, including articles, white papers, webinars, and events, it also excels in vendor-agnostic online learning, organization level assessments and trainings, and certifications. - DataFramed
DataFramed is a weekly podcast that explores how technology and data are changing the world. The co-hosts, Adel Nehme and Richie Cotton, invite data leaders and practitioners to share their insights and experiences into how they lead the charge in this data age. Whether you're an aspiring data professional seeking career guidance, an active practitioner needing updates on the newest trends and tools, or a leader aiming to reshape your organization's data use, this podcast has something for everyone. - Benn.Substack
A Substack newsletter by Benn Stancil that offers an interesting look at how data and technology are shaping almost everything in our worlds. His writing resonates not only with folks who are about ‘all-things-data’ but anyone seeking an informative and entertaining take on data related topics. Stancil has a knack for breaking down complex concepts into compelling narratives that everyone can appreciate. Check his content if you want insightful writing on tech, business, and society from a knowledgeable yet relatable perspective. - Data Products
Data Products is a newsletter by Chad Sanderson, who is a data leader. He writes about data products, data contracts, data modeling, and the future of data engineering and data architecture. His content is clear, concise and so very informative. He also leads a growing Data Quality community on slack that is open to everybody and has channels like #data_modeling, #data_memes, #questions_answered, #women_and_woman-identifying, #hiring-or-searching, etc. I find it a great forum to learn from others, share knowledge, and keep up with the latest trends.
But wait, there’s one more. A bonus resource that deserves your attention -
It’s not just another informational hub, it uncovers innovative strategies that spark ‘aha’ moments. It features actionable tutorials, technical analysis, industry trends, interviews and even mindfulness. It’s different because it debunks myths such as:
-
This is only for data engineering
- An ‘all-in-one’ tool can solve all data problems
- Modern Data Stack is the solution
- Building is better than buying
While this may seem like a plug for TimeXtender (which technically it is), the goal is to inform, not sell. As a reader, you can expect transparent content that is as informative as it is entertaining. Check out this blog post for starters.
best practices
All this knowledge can become a wasted arsenal if not applied properly. Knowing how to act upon your understanding is equally important, if not more. This is where tried-and-true strategies guide your efforts and help overcome common pitfalls.
No matter where you stand in your data management journey, the following best practices are timeless.
- Rationalizing Data
Redundancy Control and Quality Assurance
What’s even the point of having multiple entries for the same customer transaction? Why store every piece of data, regardless of its relevance?
It's required to have periodic audits to identify and eliminate duplicate data for cleaner data repositories, optimized storage costs, and accurate analytics. Data accuracy is essential to prevent the creation of a costly “data swamp” that houses irrelevant and low-quality data. Organizations need good data that is ripe for analysis. - Securing Data
Access Control and Encryption
In today’s interconnected world, privacy remains as one of the most critical concerns. Going a level above from products and services, people deal with people. This translates to storing and preventing the misuse of sensitive data like social security numbers, credit card details, proprietary algorithms, patents, copyrights, litigation documents, etc.
The organizations need to have a robust layer(s) of security and access control mechanisms to ensure different levels of authorization to different sets of data. - Preserving Data
Storage and Recovery
According to a report from the Federal Emergency Management Agency (FEMA), 40% of businesses do not reopen following a disaster. On top of that, another 25% fail within one year.
For organizations, unplanned downtime can lead to a huge unplanned expense. Long-term data preservation and immediate data recovery are both essential aspects of data management. Organizations can consider different storage methods like cloud, networked drives, external drives based on their specific requirements. Additionally, a data-recovery strategy is vital to ensure business continuity in case of a bleak event. This could include the popular 3-2-1 methodology (3 copies of data, 2 types of storage, 1 copy offsite). - Organizing Data
Documentation and Metadata
From the perspective of a user, if it [data, feature, transaction.. ] is not documented, it does not exist, and if it’s documented incorrectly, it's broken.
I cannot stress enough how important it is for organizations to have an in-depth documentation and comprehensive metadata management. It's for the efficient data utilization, data discoverability, long-term accessibility, data governance and understanding of the data lineage. It's for the “single version of the truth”. Some of the effective ways to achieve this are practicing data cataloging, standardizing naming conventions and incorporating descriptive metadata for data sets. - Cultivating a Data Culture
Engagement and Expansion
This starts at the top, where leaders show the importance of data in decision-making. They ask the right questions and show openness to answers that data provides, even if those answers challenge established beliefs (box #2 from The Three Box Solution). By making data easily available and having 'data champions' in all teams, everyone can be data-conscious. A data culture takes us from guessing to knowing, sterile numbers to engaging narratives and curiosity to innovation. - Investing in the Right Tool
Holistic and Unified
Every organization has specific set of data needs, and so, the chosen software(s) should cohesively manage all aspects of data. It needs to offer much more than ingestion, transformation, and modeling. It needs to act as a democratizing force that breaks down silos and enables users at every level. It needs to offer a strong data lineage. It needs to maintain your data’s health in the form of audits, cleansing, and validations.
The best of such tools take a holistic approach, using metadata to unify each layer of the data stack and automating manual processes. They focus on agility over fragmentation, which leads to speedier construction of data solutions and a significant reduction in costs.
Investing in a tool isn’t just about buying it, it’s about changing the way your organization handles its most valuable asset - data. A fitting example of such an investment is TimeXtender. It’s your ally, embodying the principle that time matters. While valuing your time, it enables you with quicker and accurate data-driven business insights.
Summary
- Provides a list of valuable resources for staying updated with data management trends and best practices.
- The resources range from comprehensive portals, publications, newsletters, podcasts, to user forums, covering diverse topics fin the data space.
- Each resource is explained briefly, highlighting the unique features and value they offer to the readers.
- The blogpost emphasizes the importance of applying the knowledge properly and lists timeless best practices in data management.
- The practices include rationalizing data, securing data, preserving data, organizing data, cultivating a data culture, and investing in the right tool.
- The blogpost ends by emphasizing the importance of a holistic data management tool, exemplifying with TimeXtender, that can provide quick and accurate data-driven business insights.