snhu DAT 260 Module Three Assignment Template 0 Big Data Analysis Tools Complete this template by replacing the bracketed text with the relevant information. I. Tool Comparison Table
Tool Hive Strengths • Organization: Includes different views that allow users to see different projects automated easily. Also helps workflows, arrange tasks and attributes like are annoyed to files, dates, budgets, and assignees to keep things organized. · Visualization: Includes the option of visualizations like Gantt chart, calendar, table, and kanban board as well as the ability to easily switch between them. · Monitoring: Includes dashboards that show the productivity of a team. It can send real-time notifications about performance anomalies as well as the ability to make suggestions to improve productivity. snhu Weaknesses · Task Dependencies: Even though Hive says it can allow project managers frequently have to manually create dependent tasks. • Notifications: Hive sends notifications as a pop-up not allowing them to be marked as read and allows some users to mute notifications. · User Navigation: Different users of Hive state that using it is a challenge and time-consuming to adapt to. They also say some functions are hard to understand and learn such as configuring and personalizing the software. Best Used • Can be ideal for large- scale data processing, log analysis, and social media analytics as well as well suited for batch processing scenarios. · Three companies that use Hive are Ibotta, Monday.com, and Liftoff. • Ibotta uses it to allow marketers to influence purchasing decisions and track customer behavior. · Monday.com utilizes collaboration and visualization within Hive to create fast, beautiful, and responsive work. • Liftoff uses Hive for its collaboration and inclusivity to help support mobile businesses.
Spark · Customizable: Allows users to choose from multiple languages such as Python, R, Scala, and Java as well as the ability to create testable custom code. · Easy to use: Allows for "Lazy Execution" that enables you to define complex transformation. It offers easy conversions with its "toPandas()" method and easy transformations with its "groupBy" and "map-side join" options. · Open-Source Community: "Spark has a massive open- source community behind it. The community improves the core software and contributes practical add-on packages. For example, a team has developed a natural language processing library for Spark." (thenewstack.io) snhu · Difficult to tune and maintain: If cluster is not managed well then Spark can buckle easily under heavy data science workloads. · Debugging: Even though the "DataFrame" operation catches some bugs, memory errors and errors that occur in user- defined functions are difficult to catch. • Slow UDFs and difficulty ensuring parallelism: UDFs in PySpark "are much slower and more memory- intensive than Scala and Java UDFs are." (thenewstack.io) Although one of Sparks ' pros is distributed computation, it can be difficult to make sure it is parallelizing computations as much as possible. This can lead to there not being enough chunks of work for each executor or there being too much for each and · Real-Time Data: Best