A frontend for the ingestion process.

[DevashishRaghav]

Hello cQube Community,

We're excited to kick off a new discussion on a topic that can greatly enhance the user experience and efficiency of cQube - developing a frontend for the ingestion process.

The Current Scenario

As many of you are aware, cQube is a comprehensive platform that allows collection, processing, and visualization of data effectively. One of the critical steps in this process is data ingestion, where raw data is transformed and prepared for analysis.

Currently, this ingestion process is typically managed through backend scripts and tools. While this approach has served us well, we believe that introducing a dedicated frontend for ingestion can offer several advantages.

The Proposal

We propose the development of a user-friendly frontend interface that will allow users to:

• Easily Configure Data Sources: Users can select and configure various data sources, including file uploads, databases, APIs, and more, all through a user-friendly interface.

• Monitor Ingestion Progress: Real-time progress bars and status updates will keep users informed about the ingestion process, reducing uncertainty and wait times.

• Schedule and Automate: Users can schedule data ingestion jobs and set up automation rules to streamline repetitive tasks.

Your Insights Matter

Before we proceed with the development of this frontend, we want to hear from you:

• Have you encountered challenges or bottlenecks in the current data ingestion process?
• What features or functionalities would you expect from a frontend for data ingestion?
• Do you have any best practices or lessons learned from similar projects?

Join the Discussion

Your participation in this discussion is crucial. We invite you to share your thoughts, experiences, and suggestions on this topic. Your input will guide us in creating a frontend that truly meets the needs of the cQube community.

Please feel free to comment below with your insights, or if you have any technical details or resources to share, they would be greatly appreciated.

Let's work together to make data ingestion in cQube more accessible and efficient. Thank you for your continued support and collaboration.

Thank you!

@rhwarrier @Sunbird-cQube/sunbirdcqube_architectureworkinggroup