Stream Lookup

Description

The Stream lookup step type allows you to look up data using data coming from other steps in the transformation. The data coming from the Source step is first read into memory and is then used to look up data from the main stream.
In the example below, the transformation adds data coming from a text-file(i.e., text-file should be sorted)  (B) to data coming from a database table (A):

Data from B is used to perform the lookups as indicated by the Source step option shown below.

Note: Consider using the Database Lookup step when the Lookup step originates from a table. In this case, it is faster to use the Database Lookup step and enable the option
Load all data from tablethat pre-loads the cache.

Options

The table below describes the features available for configuring the stream lookup:

Option	Description
Step name	Name of the step this name has to be unique in a single transformation
Lookup step	The step name where the lookup data is coming from
The keys to lookup...	Allows you to specify the names of the fields that are used to look up values. Values are always searched using the "equal" comparison
Fields to retrieve	You can specify the names of the fields to retrieve here, as well as the default value in case the value was not found or a new field name in case you didn't like the old one.
Preserve memory	Encodes rows of data to preserve memory while sorting
Key and value are exactly one integer field	Preserves memory while executing a sort
Use sorted list	Enable to store values using a sorted list; this provides better memory usage when working with data sets containing wide rows
Get fields	Automatically fills in the names of all the available fields on the source side (A); you can then delete all the fields you don't want to use for lookup.
Get lookup fields	Automatically inserts the names of all the available fields on the lookup side (B). You can then delete the fields you don't want to retrieve