Data Management#
This part of the tutorial provides a comprehensive overview of the process for importing data into the ConverSight platform, including the creation of datasets for analysis, configuration of SME (subject matter experts) and establishment of data governance. By the end of this tutorial, you will have a clear understanding of the steps required to get your data ready for analysis on the ConverSight platform.
Data Lake#
A data lake is a centralized repository that is used to store large amounts of structured, semi-structured and unstructured data in its raw format.
Advantages of Data Lake:
Scalability: Data lakes can store large amounts of data, making them well-suited to machine learning projects that require large datasets.
Flexibility: Data lakes allow for the storage of data in its raw format, making it easy to store and access data from a variety of sources, including databases, log files and social media.
Speed: Data lakes can support fast and efficient data retrieval, which can be important for machine learning projects that require real-time data processing and analysis.
Integration: Data lakes can be easily integrated with other tools and technologies, such as machine learning algorithms and data analytics tools, making it easier to build and deploy machine learning models.
Cost-effective: Data lakes can be more cost-effective than traditional data storage solutions, as they do not require the same level of data preparation or indexing.
By using a data lake to store data, organizations can create a centralized repository that supports the development, deployment and management of machine learning models. This can help to improve data quality and accuracy and enable organizations to make more informed decisions.
Data Storage#
Data storage in ConverSight can be achieved through a variety of methods, including databases, cloud storage solutions or file systems. The choice of storage method will depend on the specific requirements of the platform, such as the amount of data to be stored, the required processing speed and the desired level of security and reliability.
Initially the data’s collected through the data collectors will be stored in ConverSight database.
Examples of databases commonly used in conversational platforms include relational databases such as MySQL or PostgreSQL and NoSQL databases like MongoDB. Cloud storage solutions, such as Amazon S3 or Google Cloud Storage, can also be used to store data in ConverSight. These solutions offer scalability, reliability and durability, making them ideal for storing large amounts of data in the cloud.
Data Governance and Security#
Data governance is crucial for ensuring the proper management and protection of data in ConverSight. Effective data governance practices maintaining data privacy and security, improving data quality and providing clear roles and responsibilities for those work with the data. This part is achieved by the Data Administrator.
The role of a data administrator is to ensure the effective and efficient operation of an organization’s data systems and to ensure that data is accurate, secure and available to authorized users.
The data administrator has the authority to grant dataset access to others by following the steps:
Step 1: Navigate to the menu, Data Governance.
Data Governance#
Step 2: You can see a list of people you can grant access to. When you click the dataset access icon under the actions tab, you will be taken to a page where you can choose which dataset you want to grant access to.
Dataset Access#
Step 3: Here, you may choose the dataset and specify which data and columns the user should see in the dataset.
a) You can grant access to a user for a certain piece of data by choosing the roles.
b) You can grant access to a user for a particular column by choosing the group.
You can create new access for as many users as you want by selecting the new role and new group options.
Roles and Groups#
Object Models#
The object model provides a way to structure and organize the data of a specific domain in a template (model), making it easier to analyze and observe similarly structured entities.
Let’s say a dataset from the sales domain comprises information such as discounts, payment information, sales order items and so on and we did data engineering on the same and stored it as a template. When a new sales dataset is introduced, the existing model significantly reduces the amount of effort required on the new dataset and helps us provide accurate and quicker insights.
Using a model can also ensure consistency and standardization, helping to minimize errors and improve the quality of the output.
Forms#
ConverSight platform provides another excellent feature to help manage your business better. Forms is a way to collect accurate sales and inventory data from users and prepare a purchase order to replenish the business’ inventory with great ease.
For example, a form may consist of text fields, drop-down menus or checkboxes and may be used to gather information such as the type of product being sold, the customer’s location or the date of a transaction.
The information collected through forms is then used as input for machine learning algorithms, which can analyze the data to identify patterns and make predictions about future events. Forms are a flexible and user-friendly way to gather information and are used in a wide range of applications, from customer relationship management to predictive maintenance.
Improved accuracy: Automated ordering systems can ensure that orders are placed accurately and that the correct product and quantity is ordered, reducing the risk of errors and improving customer satisfaction.
Better inventory management: Automated ordering systems can be integrated with inventory management systems, making it easier to track inventory levels and to automate the reordering process when inventory levels are low.
Increased customer satisfaction: Automated ordering systems can provide customers with a convenient and easy-to-use ordering process, improving customer satisfaction and increasing the likelihood of repeat business.
Increased sales: Automating the ordering process can help businesses process more orders in less time, increasing sales and revenue.
Increased efficiency: Automating the ordering process can significantly speed up the time it takes to process an order, reducing the need for manual data entry and reducing the risk of errors.