Skip to content

Dragonfly doc updates #740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
bfc5b26
Ingestion article
SolomiiaSnihur Apr 1, 2025
5992389
Manual data entry section
SolomiiaSnihur Apr 1, 2025
119f2f9
Terminology changes
SolomiiaSnihur Apr 2, 2025
6e39820
Monitoring updates
SolomiiaSnihur Apr 2, 2025
f5cf51f
Updated change date in Monitoring
SolomiiaSnihur Apr 2, 2025
d8ec0bd
Preview article update
SolomiiaSnihur Apr 3, 2025
84fcb0c
Source records approval article
SolomiiaSnihur Apr 3, 2025
8b6c30a
Changed order in Additional operations
SolomiiaSnihur Apr 3, 2025
6a52888
Approval video link
SolomiiaSnihur Apr 3, 2025
2b779b4
Validations article
SolomiiaSnihur Apr 3, 2025
1c36c18
Entity type > business domain change
SolomiiaSnihur Apr 9, 2025
6951270
Entity type > Business domain change in progress
SolomiiaSnihur Apr 9, 2025
c180f10
Added Operations on source records video link
SolomiiaSnihur Apr 9, 2025
0330446
Added Validations video
SolomiiaSnihur Apr 11, 2025
636c256
Terminology change in progress
SolomiiaSnihur Apr 11, 2025
5039d90
Terminology changes in progress
SolomiiaSnihur Apr 11, 2025
84de11c
Manual data entry update
SolomiiaSnihur Apr 17, 2025
8447eac
Release article and Powef Fx article
SolomiiaSnihur Apr 17, 2025
925b288
Added Ingestion dashboard video
SolomiiaSnihur Apr 18, 2025
e713671
Search article update
SolomiiaSnihur Apr 18, 2025
3f0ba76
Added Stream logs article
SolomiiaSnihur Apr 18, 2025
3dd0098
Updated the Access control section
SolomiiaSnihur Apr 18, 2025
f50331c
Updated rules reference (mask action)
SolomiiaSnihur Apr 18, 2025
2daff32
Updated Additional operations article
SolomiiaSnihur Apr 18, 2025
8a01da4
Removed ingestion reports from docs
SolomiiaSnihur Apr 23, 2025
1fd3b62
Terminology change in progress
SolomiiaSnihur Apr 23, 2025
c0001b1
Updated terms and screenshots in Review mapping
SolomiiaSnihur Apr 24, 2025
19f70c0
Code to identifier terminology change in progress
SolomiiaSnihur Apr 24, 2025
df6ce8a
Terminology change: replaced screenshots in enricher docs
SolomiiaSnihur Apr 25, 2025
37e8f29
Terminology change in progress
SolomiiaSnihur Apr 25, 2025
e701958
Terminology update: replacing diagrams
SolomiiaSnihur Apr 29, 2025
ea0b634
Terminology update in progress
SolomiiaSnihur Apr 29, 2025
f9e9a6a
Replaced a diagram in Origin
SolomiiaSnihur Apr 30, 2025
00999c7
Added Azure Open AI enricher doc
SolomiiaSnihur Apr 30, 2025
fcd9ffa
Updated enricher reference
SolomiiaSnihur Apr 30, 2025
dae3912
Updated video links
SolomiiaSnihur Apr 30, 2025
2168583
Added link to manual data entry video
SolomiiaSnihur Apr 30, 2025
0bbe2ab
Updated getting started with hierarchies, glossary, relations
SolomiiaSnihur May 13, 2025
47aeb2d
Replaced training data file for getting started
SolomiiaSnihur May 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-cleaning/find-data-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-cleaning/find-data-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-cleaning/find-data-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-cleaning/find-data-4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-cleaning/modify-data-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-ingestion/create-mapping-7.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-ingestion/process-data-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-ingestion/process-data-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/getting-started/data-streaming/create-stream-2.png
Binary file modified assets/images/getting-started/deduplication/dedup-2.png
Binary file modified assets/images/getting-started/glossary/create-term-1.png
Binary file modified assets/images/getting-started/glossary/create-term-4.png
Binary file modified assets/images/getting-started/glossary/manage-glossary-1.png
Binary file modified assets/images/getting-started/glossary/manage-glossary-2.png
Binary file modified assets/images/getting-started/relations/add-edge-1.png
Binary file modified assets/images/getting-started/relations/entity-mapping-2.png
Binary file modified assets/images/getting-started/relations/view-relations-1.png
Binary file modified assets/images/getting-started/relations/view-relations-2.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-2.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-5.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-6.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-7.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-8.png
Binary file modified assets/images/getting-started/rule-builder/rule-builder-9.png
Binary file modified assets/images/integration/data-sources/review-mapping-1.png
Binary file modified assets/images/integration/data-sources/review-mapping-2.png
Binary file modified assets/images/integration/data-sources/review-mapping-3.png
Binary file modified assets/images/integration/data-sources/review-mapping-4.png
Binary file modified assets/images/integration/data-sources/review-mapping-5.png
Binary file modified assets/images/key-terms-and-features/codes-1.gif
Binary file modified assets/images/key-terms-and-features/codes-2.png
Binary file modified assets/images/key-terms-and-features/codes-3.png
Binary file modified assets/images/key-terms-and-features/codes-4.png
Binary file modified assets/images/key-terms-and-features/codes-merge-1.gif
Binary file modified assets/images/key-terms-and-features/entity-origin-code.png
Binary file modified assets/images/key-terms-and-features/linking-golden-records.png
Binary file modified assets/images/key-terms-and-features/merging-by-codes-2.png
Binary file modified assets/images/playbooks/codes-duplicates.png
Binary file modified assets/images/playbooks/configure-mapping.png
Binary file modified assets/images/preparation/enricher/clearbit-enricher-2.png
Binary file modified assets/images/preparation/enricher/duck-duck-go-enricher-2.png
Binary file modified assets/images/preparation/enricher/gleif-enricher-5.png
Binary file modified assets/images/preparation/enricher/libpostal-enricher-2.png
Binary file modified assets/images/preparation/enricher/permid-enricher-2.png
Binary file modified assets/images/preparation/enricher/vatlayer-enricher-2.png
Binary file modified assets/images/preparation/enricher/web-enricher-2.png
11 changes: 11 additions & 0 deletions assets/other/training-company.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
company_id,company_name
1,Brown-Nitzsche
2,Green-Gleason
3,Runolfsson and Sons
4,"Schoen, Bashirian and Roob"
5,Schmidt-Rohan
6,Boehm-Mayert
7,Ratke-McLaughlin
8,Goldner Inc
9,"Reichert, Parisian and Torphy"
10,Monahan Group
30 changes: 15 additions & 15 deletions assets/other/training-data.csv
Original file line number Diff line number Diff line change
Expand Up @@ -37,24 +37,24 @@ id,first_name,last_name,email,job_title
36,Lukas,Greenwood,[email protected],General Manager
37,Sindee,Gotcliff,[email protected],Database Administrator
38,Mersey,Aspin,[email protected],Database Administrator
39,Alis,Baly,abaly12@salon.com,Database Administrator
39,Alis,Baly,abaly12@dropbox.com,Database Administrator
40,Ervin,Tann,[email protected],Acountant
41,Glad,Formilli,[email protected],Acountant
42,Guthry,De Stoop,[email protected],Acountant
43,Alissa,Fearon,[email protected],Acountant
44,Sibella,Preston,[email protected],Acountant
45,Davidde,Scamaden,dscamaden18@bloglovin.com,Acountant
45,Davidde,Scamaden,dscamaden18@dropbox.com,Acountant
46,Keefe,Purdom,[email protected],Automation Specialist
47,Ward,Leaman,[email protected],Automation Specialist
48,Eberhard,Francesc,[email protected],Recruiter
49,Denni,Laye,dlaye1c@wix.com,Account Executive
49,Denni,Laye,dlaye1c@dropbox.com,Account Executive
50,Filberto,Regi,[email protected],Account Executive
51,Lamond,Acosta,[email protected],VP Sales
52,Nance,Tween,[email protected],Assistant Manager
53,Haily,Lesper,hlesper1g@t-online.de,Assistant Manager
53,Haily,Lesper,hlesper1g@dropbox.com,Assistant Manager
54,Gustavo,McPeck,[email protected],Assistant Manager
55,Zack,Cauderlie,[email protected],Assistant Manager
56,Sloan,Pinfold,spinfold1j@tinypic.com,Help Desk Operator
56,Sloan,Pinfold,spinfold1j@dropbox.com,Help Desk Operator
57,Dion,Feldfisher,[email protected],Help Desk Operator
58,Franchot,Kelshaw,[email protected],Senior Quality Engineer
59,Merla,Benallack,[email protected],Senior Quality Engineer
Expand All @@ -65,14 +65,14 @@ id,first_name,last_name,email,job_title
64,Jsandye,Satchell,[email protected],Senior Editor
65,Lauree,Vauls,[email protected],Senior Editor
66,Troy,Raittie,[email protected],Product Manager
67,Ertha,Doelle,edoelle1u@salon.com,Product Manager
67,Ertha,Doelle,edoelle1u@dropbox.com,Product Manager
68,Gracie,Vigours,[email protected],Product Manager
69,Sallee,Disdel,[email protected],Help Desk Operator
70,Way,Leet,[email protected],Information Systems Manager
71,Sammy,Laughrey,[email protected],Information Systems Manager
72,Kristina,Taffs,[email protected],Help Desk Operator
73,Abe,MacGilmartin,[email protected],Help Desk Operator
74,Yvonne,Marder,ymarder21@photobucket.com,Help Desk Operator
74,Yvonne,Marder,ymarder21@dropbox.com,Help Desk Operator
75,Rachel,Bulcock,[email protected],Help Desk Operator
76,Jamey,Monelle,[email protected],Help Desk Operator
77,Kyle,Orans,[email protected],Accounting Assistant
Expand All @@ -81,12 +81,12 @@ id,first_name,last_name,email,job_title
80,Wallache,Surman,[email protected],Accounting Assistant
81,Josiah,Legat,[email protected],Accounting Assistant
82,Karla,Spykins,[email protected],Software Developer
83,Georas,Nehls,gnehls2a@example.com,Software Developer
83,Georas,Nehls,gnehls2a@dropbox.com,Software Developer
84,Hewie,Tremmil,[email protected],Software Developer
85,Cristobal,Broggini,[email protected],Project Manager
86,Leilah,Parnaby,[email protected],Project Manager
87,Mureil,Groger,[email protected],Legal Assistant
88,Irwinn,Meehan,imeehan2f@cbslocal.com,Legal Assistant
88,Irwinn,Meehan,imeehan2f@dropbox.com,Legal Assistant
89,Rafferty,Goodings,[email protected],Legal Assistant
90,Mathias,Matschoss,[email protected],Quality Engineer
91,Anna-maria,Petrakov,[email protected],Account Representative
Expand All @@ -96,25 +96,25 @@ id,first_name,last_name,email,job_title
95,Farand,Elfitt,[email protected],Accountant
96,Earvin,Tash,[email protected],Accountant
97,Jenilee,Fishly,[email protected],Research Assistant
98,Beckie,Martinson,bmartinson2p@symantec.com,Research Assistant
98,Beckie,Martinson,bmartinson2p@dropbox.com,Research Assistant
99,Lilla,Kingsworth,[email protected],Research Assistant
100,Kyla,Ferreri,[email protected],Marketing Assistant
101,Eddy,Shuard,[email protected],Marketing Assistant
102,Nels,Bembrick,[email protected],Marketing Assistant
103,Marshal,Calverley,[email protected],Marketing Assistant
104,Jacquenetta,Sparshott,[email protected],Marketing Assistant
105,Wesley,Volett,[email protected],Software Test Engineer
106,Hesther,Hamflett,hhamflett2x@usa.gov,Software Test Engineer
106,Hesther,Hamflett,hhamflett2x@dropbox.com,Software Test Engineer
107,Halli,Predohl,[email protected],Software Test Engineer
108,Dominique,Wikey,[email protected],Software Test Engineer
109,Basil,Ganning,[email protected],Software Test Engineer
110,Berny,Duke,[email protected],Technical Writer
111,Vinny,Sprowles,vsprowles32@csmonitor.com,Technical Writer
111,Vinny,Sprowles,vsprowles32@dropbox.com,Technical Writer
112,Aldrich,Jendricke,[email protected],Technical Writer
113,Odessa,Horsley,[email protected],Technical Writer
114,Tine,Guillond,[email protected],Technical Writer
115,Mahala,Hamshar,[email protected],Computer Systems Analyst
116,Matilde,Lemme,mlemme37@shinystat.com,Computer Systems Analyst
116,Matilde,Lemme,mlemme37@dropbox.com,Computer Systems Analyst
117,Kimberley,Tiffney,[email protected],Computer Systems Analyst
118,Evie,Mostin,[email protected],Computer Systems Analyst
119,Kristel,Warrell,[email protected],Computer Systems Analyst
Expand All @@ -123,8 +123,8 @@ id,first_name,last_name,email,job_title
122,Esra,Brevetor,[email protected],Senior Quality Engineer
123,Korney,Stych,[email protected],Senior Quality Engineer
124,Bea,Dottridge,[email protected],Senior Quality Engineer
125,Paton,Duggan,pduggan3g@example.com,Senior Quality Engineer
126,Anthe,O'Cooney,aocooney3h@desdev.cn,Senior Quality Engineer
125,Paton,Duggan,pduggan3g@dropbox.com,Senior Quality Engineer
126,Anthe,O'Cooney,aocooney3h@dropbox.com,Senior Quality Engineer
127,Gianina,Farrey,[email protected],Senior Quality Engineer
128,Erina,Borton,[email protected],Sales Representative
129,Yvon,Cutforth,[email protected],Sales Representative
Expand Down
21 changes: 21 additions & 0 deletions assets/other/training-employee.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
employee_id,first_name,last_name,company_id
1,Binni,Lindblom,2
2,Aldric,Green,2
3,Thaxter,Geale,2
4,Dyanne,Scotchmore,2
5,Gustavo,Fox,5
6,Iago,Latour,5
7,Hynda,Bertholin,7
8,Gayelord,Mapes,7
9,Miller,Bunner,7
10,Coralyn,Durbyn,7
11,Lonnie,Kield,8
12,Sheree,Daines,8
13,Nan,Eastby,8
14,Benito,Yurocjhin,10
15,Klarrisa,Ianne,10
16,Marylynne,Docwra,10
17,Armin,Mallabar,10
18,Alexia,Athow,10
19,Ava,Gullen,10
20,Lyssa,Darlasson,10
20 changes: 9 additions & 11 deletions docs/010-getting-started/020-data-ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Ingesting data to CluedIn involves three basic steps: importing, mapping, and pr
<iframe src="https://player.vimeo.com/video/843840937?badge=0&amp;autopause=0&amp;player_id=0&amp;app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture" allowfullscreen title="Getting started with data ingestion in CluedIn"></iframe>
</div>

In this guide, you will learn how to import a file into CluedIn, create a mapping, process the data, and perform data searches.
In this guide, you will learn how to import a file into CluedIn, create a mapping, process the data, and search for golden records.

**File for practice:** <a href="../../../assets/other/training-data.csv" download>training-data.csv</a>

Expand All @@ -29,9 +29,9 @@ A CSV (comma-separated values) file format allows data to be saved in a tabular

**To import a file**

1. On the home page, in the **Integrations** section, select **Import From Files**.
1. On the navigation pane, go to **Ingestion**, and then in the **Files** section, select **Add**.

![import-a-file-1.png](../../assets/images/getting-started/data-ingestion/import-a-file-1.png)
![files-add.png](../../assets/images/getting-started/data-ingestion/files-add.png)

1. In the **Add Files** section, add the file. You may drag the file or select the file from the computer.

Expand All @@ -49,9 +49,7 @@ After you uploaded the file, you can view the data from the file as a table with

**To view imported data**

1. On the home page, in the **Integrations** section, select **View All Data Sources**.

Alternatively, on the navigation pane, go to **Integrations** > **Data Sources**.
1. On the navigation pane, go to **Ingestion** > **Sources**.

1. Find and expand the group that you created in the previous procedure.

Expand Down Expand Up @@ -89,11 +87,11 @@ Mapping is the process of creating a semantic layer for your data so that CluedI

1. Select **Next**.

1. In **Entity Type**, enter the name of a new entity type and select **Create**. An entity type is a specific business object within the organization. A well-named entity type is global and should not be changed (for example, Person, Organization, Car) across sources.
1. In **Business Domain**, enter the name of a new business domain and select **Create**. A business domain is a specific business object within the organization. A well-named business domain is global and should not be changed (for example, Person, Organization, Car) across sources.

The **Entity Type Code** is created automatically; it is a string that represents the entity type in code (for example, in clues).
The **Business Domain Identifier** is created automatically; it is a string that represents the business domain in code (for example, in clues).

1. In **Icon**, select the visual representation of the entity type.
1. In **Icon**, select the visual representation of the business domain.

![create-mapping-3.png](../../assets/images/getting-started/data-ingestion/create-mapping-3.png)

Expand All @@ -103,7 +101,7 @@ Mapping is the process of creating a semantic layer for your data so that CluedI

![create-mapping-4.png](../../assets/images/getting-started/data-ingestion/create-mapping-4.png)

1. In **Origin**, make sure that the selected field is appropriate for generating a unique identifier (Entity Origin Code) for each record.
1. In **Primary Identifier**, review the field that was selected automatically for generating a unique identifier for each record.

![create-mapping-5.png](../../assets/images/getting-started/data-ingestion/create-mapping-5.png)

Expand Down Expand Up @@ -131,7 +129,7 @@ Processing turns your data into golden records that can be cleaned, deduplicated

1. Select **Process**.

1. Review information about records that will be processed. Pay attention to the **Origin Entity Code Status** and **Code Status** sections. If there are duplicates, they will be merged during processing.
1. Review information about records that will be processed. Pay attention to the **Primary Identifier Status** section. If there are duplicates, they will be merged during processing.

![process-data-2.png](../../assets/images/getting-started/data-ingestion/process-data-2.png)

Expand Down
12 changes: 7 additions & 5 deletions docs/010-getting-started/030-manual-data-cleaning.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,19 +35,21 @@ Finding the data that needs to be cleaned involves defining search filters and s

1. In the search field, select the search icon. Then, select **Filter**.

1. In the **Entity Types** dropdown list, select the entity type to filter the records.
1. In the **Business Domains** dropdown list, select the business domain to filter the records.

![find-data-1.png](../../assets/images/getting-started/data-cleaning/find-data-1.png)

As a result, all records with the selected entity type are displayed on the page. By default, the search results are shown in the following columns: **Name**, **Entity Type**, and **Description**.
As a result, all records with the selected business domain are displayed on the page. By default, the search results are shown in the following columns: **Name**, **Business Domain**, and **Description**.

1. To find the specific values that you want to fix, add the corresponding column to the list of search results:

1. In the upper-right corner, select **Column Options**.

1. Select **Add columns** > **Vocabulary**.

1. In the search field, enter the name of the vocabulary and start the search. In the search results, select the needed vocabulary key.
1. In the **Vocabulary Keys** section, expand the vocabulary that contains the needed vocabulary key, and then select the checkbox next to it.

1. Move the vocabulary key to the **Selected Vocabulary Keys** section using the arrow pointing to the right.

![find-data-2.png](../../assets/images/getting-started/data-cleaning/find-data-2.png)

Expand All @@ -71,7 +73,7 @@ After you have found the data that needs to be cleaned, create a clean project.

**To create a clean project**

1. In the upper-right corner of the search results page, select the ellipsis button, and then select **Clean**.
1. In the upper-right corner of the search results page, open the three-dot menu, and then select **Clean**.

![create-a-clean-project-1.png](../../assets/images/getting-started/data-cleaning/create-a-clean-project-1.png)

Expand Down Expand Up @@ -101,7 +103,7 @@ After you have found the data that needs to be cleaned, create a clean project.

![modify-data-2.png](../../assets/images/getting-started/data-cleaning/modify-data-2.png)

1. In the upper-right corner, select **Process**. In the confirmation dialog box, select **Skip stale data** and clear the **Enable rules auto generation** checkbox. Then, confirm that you want to process the data.
1. In the upper-right corner, select **Process**. In the confirmation dialog box, select **Skip** and leave the **Enable automatic generation of data part rules** checkbox cleared. Then, confirm that you want to process the data.

![modify-data-3.png](../../assets/images/getting-started/data-cleaning/modify-data-3.png)

Expand Down
27 changes: 7 additions & 20 deletions docs/010-getting-started/040-deduplication.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,35 +25,29 @@ In this guide, you will learn how to deduplicate the data that you have ingested

## Create deduplication project

As a first step, you need to create a deduplication project that allows you to check for duplicates that belong to a certain entity type.
As a first step, you need to create a deduplication project that allows you to check for duplicates that belong to a certain business domain.

**To create a deduplication project**

1. On the navigation pane, go to **Management**. Then, select **Deduplication**.

![create-dedup-project-1.png](../../assets/images/getting-started/deduplication/create-dedup-project-1.png)

1. Select **Create Deduplication Project**.

1. On the **Create Deduplication Project** pane, do the following:

1. Enter the name of the deduplication project.

1. Select the entity type that you want to use as a filter for all records.
1. Select the business domain that you want to use as a filter for all records.

![dedup-2.png](../../assets/images/getting-started/deduplication/dedup-2.png)

1. In the lower-right corner, select **Create**.

You created the deduplication project.

![dedup-3.png](../../assets/images/getting-started/deduplication/dedup-3.png)

Now, you can proceed to define the rules for checking duplicates within the selected entity type.
You created the deduplication project. Now, you can proceed to define the rules for checking duplicates within the selected business domain.

## Configure matching rule

When creating a matching rule, you need to specify certain criteria. CluedIn uses these criteria to check for matching values among records belonging to the selected entity type.
When creating a matching rule, you need to specify certain criteria. CluedIn uses these criteria to check for matching values among records belonging to the selected business domain.

**To configure a matching rule**

Expand Down Expand Up @@ -87,8 +81,6 @@ When creating a matching rule, you need to specify certain criteria. CluedIn use

The status of the deduplication project becomes **Ready to generate**.

![dedup-7.png](../../assets/images/getting-started/deduplication/dedup-7.png)

1. In the upper-right corner, select **Generate Results**. Then, confirm that you want to generate the results for the deduplication project.

{:.important}
Expand Down Expand Up @@ -130,19 +122,14 @@ The process of fixing duplicates involves reviewing the values from duplicate re

1. Review the group that will be merged and select **Next**.

1. Select an option to handle the data merging process if more recent data becomes available for the entity. Then, select **Confirm**.
1. Select an option to handle the data merging process if more recent data becomes available for the golden record. Then, select **Confirm**.

![dedup-12.png](../../assets/images/getting-started/deduplication/dedup-12.png)

{:.important}
The process of merging data may take some time.

After the process is completed, you will receive a notification. As a result, the duplicate records have been merged into one record.

You fixed the duplicate records.
The process of merging data may take some time. After the process is completed, you will receive a notification. As a result, the duplicate records have been merged into one record.

{:.important}
All changes to the data records in CluedIn are tracked. You can search for the needed data record and on the **Topology** pane, you can view the visual representation of the records that were merged through the deduplication process.
All changes to golden records in CluedIn are tracked. You can search for the needed golden record and on the **Topology** pane, you can view the visual representation of the records that were merged through the deduplication process.

## Results & next steps

Expand Down
Loading