Skip to main content

Capture filename while using indirect file

Scenario : I have 5 source files which I am planning to load using indirect file as they all are of same format and go to same target table. One requirement is to capture the source file name in the target. Is there any simple way to achieve this? The filename column is there only for file targets, not for file sources.

Solution:

Sol 1.

Effective with PowerCenter 8.5 there is an option called Add Currently Processed Flat File Name Port.
If this flat file source option is selected, the file name port will be added in the ports of the source.
To add the CurrentlyProcessedFileName port:

1. Open the flat file source definition in the Source Analyzer.

2. Click the Properties tab.

3. Select Add Currently Processed Flat File Name Port.
The Designer adds the CurrentlyProcessedFileName port as the last column on the Columns tab.
The CurrentlyProcessedFileName port is a string port with default precision of 256 characters.

4. Click the Columns tab to see your changes.
You may change the precision of the CurrentlyProcessedFileName port if you wish.

5. To remove the CurrentlyProcessedFileName port, click the Properties tab and clear the Add Currently Processed Flat File Name Port check box.

For previous versions a shell script or batch file can be used in a pre-session command task.

------- Short desc
Double click flat file source and Go to Properties tab and check "Add Currently Processed Flat File Name Port" check box. This will add a column "CurrentlyProcessedFileName" in flat file columns list. So simple, Isn't it?

or
Sol 2.
You can append Filename to the file using Shell script...

-

#!/bin/ksh

for a in $(ls *.CSV)
do
            
     fname=$( echo $a | tr Hi, You can append Filename Hi, You can append Filename )
     sed -i -e "s/$/
t$/g" $

done;
exit 0
-

This adds file name for all .CSV files. Here the delimiter is TAB. you can change the script according to your spec.











Comments

Popular posts from this blog

SQL Transformation with examples

============================================================================================= SQL Transformation with examples   Use : SQL Transformation is a connected transformation used to process SQL queries in the midstream of a pipeline . We can insert, update, delete and retrieve rows from the database at run time using the SQL transformation. Use SQL transformation in script mode to run DDL (data definition language) statements like creating or dropping the tables. The following SQL statements can be used in the SQL transformation. Data Definition Statements (CREATE, ALTER, DROP, TRUNCATE, RENAME) DATA MANIPULATION statements (INSERT, UPDATE, DELETE, MERGE) DATA Retrieval Statement (SELECT) DATA Control Language Statements (GRANT, REVOKE) Transaction Control Statements (COMMIT, ROLLBACK) Scenario: Let’s say we want to create a temporary table in mapping while workflow is running for some intermediate calculation. We can use SQL transformat...

Load the session statistics such as Session Start & End Time, Success Rows, Failed Rows and Rejected Rows etc. into a database table for audit/log purpose.

                                                                                                                                                                     ...

CMN_1650 A duplicate row was attempted to be inserted into a dynamic lookup cache Dynamic lookup error.

Scenario: I have 2 ports going through a dynamic lookup, and then to a router. In the router it is a simple case of inserting new target rows (NewRowLookup=1) or rejecting existing rows (NewRowLookup=0). However, when I run the session I'm getting the error: "CMN_1650 A duplicate row was attempted to be inserted into a dynamic lookup cache Dynamic lookup error. The dynamic lookup cache only supports unique condition keys." I thought that I was bringing through duplicate values so I put a distinct on the SQ. There is also a not null filter on both ports. However, whilst investigating the initial error that is logged for a specific pair of values from the source, there is only 1 set of them (no duplicates). The pair exists on the target so surely should just return from the dynamic lookup newrowlookup=0. Is this some kind of persistent data in the cache that is causing this to think that it is duplicate data? I haven't got the persistent cache or...