Click here to Skip to main content
15,867,453 members
Everything / ETL

ETL

ETL

Great Reads

by Pavel Sinkevich
If data source is populated manually, there is a typical problem: a mix of visually similar Cyrillic and Latin letters. This tip describes how to fix it.
by Member 11127135
ETL Testing and data-validation, dynamic SQL rule generator
by Suryakant S. Bharne
SSIS Script task to copy Table Object with data from one MSSQL server to another using Transfer SQL Server

Latest Articles

by Cinchoo
Tip to merge different CSV files into one CSV file using Cinchoo ETL
by Cinchoo
Simple fast lite CSV file reader for .NET
by Cinchoo
Quick tutorial about flattening complex, nested JSON objects using Cinchoo ETL
by Cinchoo
Tip to split large JSON file based on deeply nested array property using Cinchoo ETL

All Articles

Sort by Score

ETL 

28 Oct 2016 by Pavel Sinkevich
If data source is populated manually, there is a typical problem: a mix of visually similar Cyrillic and Latin letters. This tip describes how to fix it.
1 Feb 2018 by Member 11127135
ETL Testing and data-validation, dynamic SQL rule generator
9 Mar 2020 by #realJSOP
You could load the excel file and the database table's schema into separate ADO datatables, and then compare the schemas between the datatables. Depending on how extensive your data type discovery is during the import process, you could also...
4 Apr 2014 by CoderPanda
I am trying to programmatically get the details of a specific execution of a transformationWhat I have tried so farI get the status of the Kettle Slave Server/Carte athttp://localhost:8282/kettle/status/From this page, I can click on the desired transformation and I am taken to...
10 Jun 2014 by Amit Ghotikar
how to load data from multiple Excel sheets to multiple OLE db tables? but these sheets are having different structure.so can it possible or not?
10 Jun 2014 by Sergey Vaselenko
Look at this article[^] and download the example package.You may create SQL queries for every Excel sheet and specify a target server, database, and a table.
19 May 2015 by danoftheman
Wondering if I could get some input from anyone on a concept that I am working on at the moment.Basically I am looking to somehow be able to allow a user to input the URL of a web service and then I want to programmatically call the service and store the configuration in a SQL db to be able...
3 Nov 2015 by Dhamodharan A
Hi All,Am using Pentaho community edition 5.4.0 ,I explain My requirement very Simply,1) I have my jobs and transformation in my local windows machine and i like to execute those in my client machine ,So that i installed same Pentaho community version 5.4.0 on his machine. For Remote...
14 May 2016 by wzkmafi
SSIS For Each Loop is not accepting variable from SQL Query Task. The SQL Query select table names, which should be looped through and each passed to a Data flow task to select * from TableName and write data to a TableName.txt file Errors:1. User::obj" does not contain a valid data...
3 Feb 2017 by Faran Saleem
Hello guys,I have a little query and i need your help.I am trying to create a ETL Process on SQL Server 2005, i have not made any ETL before.What i want is data is available in a server and i want to automate the process like the ETL should run daily at a specific time and daily dump of...
3 Feb 2017 by CHill60
This is not a good question for this forum as it is too large a subject and not specific enough.I'm afraid you are going to have to persist with the tutorials and documentation you can find on line. The following information might help you though:The tool you will need to use is called...
13 Sep 2017 by Member 13409233
where can i get cognos data manager material and download file What I have tried: i have tried googling it but unable to find material with proper screenshot.
13 Sep 2017 by Patrice T
Quote: where can i get cognos data manager material and download file Try to ask IBM, it look to be their product.
8 Jan 2018 by Rajesh Pandya
You can dislable lock escalation. For disalbe lock escalation you can specify at the table level using ALTER TABLE tablename SET(lock_escalation = DISABLE). For more detail look at https://msdn.microsoft.com/en-us/library/ms190273.aspx If you disable it can result in higher memory consumption....
5 Nov 2018 by nairnishitha
We are logging custom events from application to ETW. Now need to find the best compression technique as lots of data will be written to the file and the etl file is to be send with error report to the server. How to specify the details? In System.Diagnostics.Tracing.EventSource itself, when...
5 Nov 2018 by johannesnestler
Depend's how you use it, but Maybe this helps: ETW Trace Compression (and xperf syntax refresher) | Random ASCII[^]
18 Feb 2019 by Suryakant S. Bharne
SSIS Script task to copy Table Object with data from one MSSQL server to another using Transfer SQL Server
1 Feb 2020 by Ayan Putatunda
Problem : I have a source schema (a bunch of tables) and a destination schema (a fixed number of tables with predefined columns) in postgres. I want to design a mapper-insert function which would map the source column(s) to a particular column of a table in destination schema. What I have...
1 Feb 2020 by RickZeeland
Take a look at Orange: etl-tools-for-programmers~orange[^]
8 Mar 2020 by Member 13958707
There are multiple csv files in a folder named parent with data in it and those files have the same format. I need to extract data in the csv file and insert them into a sql server table named log.once data are inserted, the csv files should move...
8 Mar 2020 by Member 13958707
I added two foreach loop containers. 1st one contains the data flow task and it is connected to a flat-file system task component. Flat file system task in the 1st foreach loop container, the source is set to the variable where the parent folder...
9 Mar 2020 by Member 13958707
I have a ssis package where it reads a folder with csv files and transform those files into sql server table. at the same time those csv files are moved to an archive folder. If there is any file with wrong extension or file format against the...
25 Apr 2021 by OriginalGriff
Read the error message: it's pretty clear: The subquery return multiple values. This is not allowed when the subquery follows =,! =, ,> = Or when it is used as an expression. Your inner SELECT returns more than one value, which isn't...
26 Apr 2021 by CHill60
If the databases are not on the same instance of SQL Server then set up a Linked server - see Create Linked Servers - SQL Server | Microsoft Docs[^] Then just use fully qualified table names including the server e.g. select [column1],[column2]...
4 Oct 2021 by Richard MacCutchan
The title says it all: Apache Spark™ - Unified Analytics Engine for Big Data[^]
28 Jan 2022 by amine gacem
Hi, I want to read .dbf files in Talend 7.3 for data integration. I really need some help about this topic because I am sure there is a way within Talend. If there is any way to do it in Java, I can at least use it in Talend as Java code. ...
5 Mar 2020 by phil.o
Maybe that can help: SQL SERVER - Import CSV File into Database Table Using SSIS - SQL Authority with Pinal Dave[^]
26 Sep 2022 by Cinchoo
Quick tutorial about flattening complex, nested JSON objects using Cinchoo ETL
12 Jan 2022 by Cinchoo
Tip to split large JSON file based on deeply nested array property using Cinchoo ETL
27 Sep 2022 by Cinchoo
Simple fast lite CSV file reader for .NET
20 Oct 2021 by Cinchoo
Quick tutorial on converting CSV file to JSON file using Cinchoo ETL
25 Feb 2018 by ShamimSiddiquee
Manage broken batch packages re-run start point.
30 Jan 2018 by Member 11127135
Oracle materialized views under ETL processes
6 Mar 2020 by MarcusCole6833
Another way to do this is with the Import Export Wizard. Right Click on the DB and got Task and import and export will appear. The MSFT documentation is below Imp/exp Wizard: Import and Export Data with the SQL Server Import and Export Wizard...
27 Oct 2021 by Cinchoo
Tip to convert JSON to XML with namespaces using Cinchoo ETL
28 Sep 2022 by Cinchoo
Tip to merge different CSV files into one CSV file using Cinchoo ETL
7 Nov 2021 by Cinchoo
Use Cinchoo ETL to deserialize selective XML nodes from large XML file
13 Dec 2021 by Cinchoo
How to compare two CSV files for Add, Deleted and Changed records using Cinchoo ETL
8 Jan 2018 by Tamanna Sharma
Hi, My question is regarding concurrency and locking mechanism. In my project, Informatica and SQL procedures have to process on a single SQL table. Process 1: Informatica dumps a set of records into SQL table X. Then SQL procedure need to take that set from X to process and then dump the...
25 Apr 2021 by houssem eddine ayari
Hi i have querry statement in my ETL to insert data in my table but i got this error : "The subquery return multiple values. This is not allowed when the subquery follows =,! =, ,> = Or when it is used as an expression." Please help !! ...
26 Apr 2021 by houssem eddine ayari
Hello, i have querry statement that selects from source and with condtion writes in destination but with different databases. What I have tried: i have made two shared connections in my project. Any help guys !