Stack Overflow
Stack Overflow

Questions tagged [database-reconciliation]

16 questions
2
votes
0 answers

What data reconciliation techniques are available for validating Debezium CDC streams?

I've been trying to find online documentation or blogs on approaches to validate end-to-end CDC capturing completeness, aka "Data Reconciliation". At my company we are using both Debezium for PG and Mongo to capture change streams and replicate them…
btiernay
  • 7,873
  • 5
  • 42
  • 48
1
vote
1 answer

Openrefine - reconcile by second or third candidate

With the reconcile service I often come across this problem: the best candidate isn't really correct, the best is the second or the third candidate (ad it has also a better score), like this: How can I select the correct one in mass? I've got…
Lara M.
  • 855
  • 2
  • 10
  • 23
1
vote
3 answers

Conditional formatting, search formula

For reference, I have two data sets, one that with scraped names and one with manually input names. The Manual input set has broader coverage but the scraped set is more accurate. So I have used textjoinif to create a list of the manually input…
Colter Miller
  • 83
  • 6
1
vote
2 answers

Mysql: set some value from an element

I my db I have some values with multiple Id and an Id with whom I reconciled them, for instance: Id: 234 | Viaf_id: 3475903 | Name: Arturo Rossi | First name: Rossi | Last name: Arturo Id: 345 | Viaf_id: 3475903 | Name: Rossi, Arturo | First name:…
Lara M.
  • 855
  • 2
  • 10
  • 23
1
vote
1 answer

SQL for validating/reconciling data across two tables, when each is a copy of the other

I am looking to validate data across 2 tables, where the TableA (in the users source database - DB2) is a copy of the TableB (in the Data warehouse - DB2/Netezza). Due to the process of copying the data across not being stable and being run on a…
gagneet
  • 35,729
  • 29
  • 78
  • 113
1
vote
1 answer

Testing Databases on the AWS Cloud (RedShift)?

We have shifted from IBM DB2 databases to having PostGRE SQL databases on the AWS Cloud. Is anyone aware of or has worked with AWS to test databases? a) If so, what tools do you use? b) What do you test when checking the databases in a Business…
gagneet
  • 35,729
  • 29
  • 78
  • 113
0
votes
0 answers

Most time-efficient way of reconciling csv/xlsx data converted between schemas?

Old Schema New Schema Id AccountId ProductId ProductCode ProductName So, colleagues of mine are converting data from an older system to a new one, which involves small changes in the schema. My job is to make sure it's reconciling…
alotofwords
  • 1
  • 1
0
votes
0 answers

Reconcile two large tables efficiently in Power BI?

As part of a migration project, I’m looking to reconcile two fact tables (high cardinality with approx 500k rows each- there are a lot of customer accounts and it has to be reconciled on a customer account basis ). There is a many-to-many…
DanAthens644
  • 1
0
votes
1 answer

Data reconciliation between 2 datasets on SQL

image_table I currently need to find all the differences between a new_master dataset and a previous one using SQL Oracle. The datasets have the same structure and consist of both integers and strings and do not have a unique key id unless I select…
Luigi
  • 59
  • 1
  • 6
0
votes
1 answer

geodata reconciliation with OpenRefine using longitude and latitude

I have a xls table containing a column with long lat information: for example "60.974387; 25.6472465" and i would like to reconcile that information in Openrefine to get the name of that place (or nearest place) Does anyone know a reconciliation…
Tim
  • 13
  • 1
0
votes
1 answer

Access, using VBA to auto match records between 2 recordsets

I have a database in Access, and another table in excel. I'm trying to build a reconciliation macro within access, that will hopefully mark all the records in Access that have a matching entry in excel. The excel will also be marked off, so I will…
Chris
  • 45
  • 1
  • 8
0
votes
1 answer

Compare data in 2 tables held in 2 different databases

I have an oracle db and a postgresql db. Both have a set of tables which have exactly the same definition. I want to compare the data in both these tables on an on-going basis. Are there any AWS services or open source tools available to do this…
Punter Vicky
  • 15,954
  • 56
  • 188
  • 315
0
votes
1 answer

Reconciliation Automation Query

I have one database and time to time i change some part of query as per requirement. i want to keep record of results of both before and after result of these queries in one table and want to show queries which generate difference. For…
Paras
  • 7
  • 6
0
votes
1 answer

sql execution through groovy

Trying to insert a row using a query. How to do reconcilation of target system using using groovy by inserting sql query? //packages import java.sql.*; import groovy.sql.Sql // class class test { //main method static void…
kumar
  • 15
  • 5
-1
votes
1 answer

Get "latest" version of customer info in Excel

I have three spreadsheets of customer data (name, email, physical address). These sheets each have a date column, representing when the contact was last updated. I need to create one spreadsheet from the three using the most recent contact info from…
Ashley Strout
  • 6,107
  • 5
  • 25
  • 45
1
2