INF30004 BUSINESS INTELLIGENCE AND DATA VISUALISATION CONTINUOUS LEARNING ACTIVITIES #1 Name: Thi Yen Nhi Tran ID: 102415004 Task 1 - the differences between data lake and data warehouse 1. Types of data The data warehouse contains data extracted from the trading system and quantitative data to aid in analysis of business performance and performance (Hoffer, Ramesh & Heikki Topi 2019, pp. 435-439). The data warehouse needs a well-structured data model that identifies the incoming data and eliminates unnecessary data. In the Data lake, all types of data from system sources are stored. Include sources of data that may be denied storage in the Data warehouse, such as web server logs, sensor data, social media activity, text and images, etc. The Data Lake can even store data that is not currently in use but may be needed in the future. This is made possible by low-cost storage solutions like Hadoop (Saurabh Gupta & Venkata Giri 2018). 2. Schema form The data warehouse applies the "Schema on Write" method, which means the model is designed for reporting purposes (Hoffer, Ramesh & Heikki Topi 2019, pp. 435-439). This process requires a significant investment of time to analyze data sources, understand business processes, classify data, and form a defined system for storing data. Data lake keeps data in its original state; When there is a need to use data to solve business problems, only relevant data are selected and analyzed to give answers. This approach is called "Schema on Read", which saves business time and money (Xuan 2020). 3. Flexibility
Because a Data warehouse is a tightly structured data warehouse, it can be costly to change the structure according to company's needs. The changing process requires a lot of complex, time consuming and costly processes. The Data Lake, on the other hand, takes advantage of the flexibility of the data, because the data is stored in raw form and is always easy to access, allowing for no-hassle refactoring. 4. User Data warehouse is familiar to businesses and users, easily responding to needs such as performance reporting, metrics, and data statistics. With a tight structure, easy to use and mainly used to answer user queries, the Data warehouse satisfies the needs of business operations. Data lake is more suitable for users who perform in-depth analysis such as data scientists. With the variety of data types in the data lake, they have the ability to combine different types of data and raise completely new questions that need to be answered. Legacy filesystem ERP Sales App ETL Warehouse Schema on Write OBIEE reporting Ad-hoc query Legacy filesystem ERP Sales App Incremental ingestion Data Lake CDC Schema on Read Pattern analysis predictive analytics - their specific applications/use Data lake applications 1. Application of data lake in oil and gas industry As one of the leading industries in the deployment and use of breakthrough technologies from cloud computing to IoT, it is not surprising that the oil and gas industry is also ahead of this new data lake trend.