Exjackly

Found this old question in the Data Direct Technologies Forum.  I think it makes a good example exercise, since it can be solved multiple ways, but is short enough to address in a single (long) post.

I have a record in this format
H,0002
L,B254,4
L,B221,19
L,B064,4
H,0001
L,B208,1
L,B186,8
H,0004
L,B178,17
L,B132,19
L,B121,17
L,B025,12

H,xxxx (xxxx represents the customer unique no)
L, yyyy.z (yyyy is the product part no, and z is the quantity. A customer could order more than one product, as a result so many Ls below H.

I want to transform this, so that the target file has the customer number(No H), the quantity of each products ordered, with the part number.

In order to get to the output column, though, the part no, of each product is checked against a part master that contain only part numbers

The reject file, must have the same format as the input file.

I have formatted the quote to make it clearer what is being sought.

My Approach

(more…)

Being a DataStage ETL developer, I have spent plenty of time over the past few years working with a tool that has proven to be both powerful and, at times, frustrating.  Powerful, in that it can handle a huge amount of data with a lot of flexibility, often with good performance, in a manner than can be maintained and expanded with a very small core of developers (even just one).  Frustration, since that power and flexibility is very sensitive to details of the implementation and error messages are often confusing or ambiguous.

When I learned DataStage, I thought the class was very easy – with the exercises very simple to get through.  The classroom exercises hid a lot of the flexibility and complexity of the tool.  Primarily, this was done by looking at each of the available stages in isolation.  Exercises were setup to build simple jobs that explored the basic functionality of the stage.

What the class did very little of was discuss maintenance of jobs, parameterization, partitioning, performance tuning and common sources of errors.  I’ve learned about those with experience.  Some of this experience was easily gained, others took a lot of time to work through and figure out and there are some things that I have not used or mastered yet.

The two best tricks that I know for developing with DataStage are as follows:
1.  Be very meticulous – especially when building many similar jobs.  Having a checklist of items for every job being built is useful.
2.  Build and test incrementally.  Adding 2 stages, compiling and testing before adding more makes development easier and troubleshooting faster.

If you have any tips for development, questions about DataStage, or comparisons with other ETL tools, I welcome your feedback.

It is well documented that setting goals is made more effective when those goals are shared with other people.

Goal setting is a good thing. I’ve known it for most of my life, even when I have not been actively practicing it. As I have gained more experience, I have found that setting goals gets harder and harder. It isn’t because the general process of setting goals has been getting more difficult. Instead, it is that my goals have been getting larger and more complex.

I am not one of those people who things that money is the scorecard of our lives. There is nothing that I do that is just for money – at least, I have convinced myself that is true. All the same, money is a significant factor in our lives. Having plenty of money makes it much easier to do a lot of things that we want to do.

Those things money lets us do are not the most important in our lives. I can still spend time with my wife and her family, even if I have no money. It is still possible to be a role model and try to teach the boys in my scout troop how to be good men no matter how much or how little money I have. There is not a toll to walk in the door at church to be a member of the community.

Admittedly, having money to spend while doing those things is an enjoyable luxury. Having the option to take a trip with my wife to spend time together and experience a new location has created many happy memories for us. Just being able to fill up the car with gas to go camping with the scouts has helped me to know the boys better.

For that reason, I have financial goals that I express in the common denominator of cash. I don’t want money for money’s sake, but because of what it will allow me to do. So, my financial goals really tie in to a number of my life goals. Here are my primary financial goals and how I am doing on each of them.

1. Net Worth – $5,000,000 in liquid assets is my goal. This amount of funds will give me enough income, without touching the principal, to meet all of my anticipated expenses through death. This includes the impact of inflation on living expenses. Even if expenses rise faster than anticipated, or a catastrophic event occurs, the principal amount will provide a significant amount of cushion.

Current Status: Approximately $125,000 in net worth

2. Own my home free and clear. The target date for this is within 10 years of moving in to a home. Currently, this goal is not being met, as we have leveraged our current home to buy a nice lot to build our next home on. We are still well into the positive equity side of the ledger however.

Current Status: Approximately $380,000 in real estate value. Loans outstanding total approximately $325,000

As you can see, I really only have 2 major financial goals. The first one incorporatesmultiple subgoals, where I will take different actions as I reach the various milestones on the way. The second refers to wanting to own my habitation free and clear and the comfort and peace of mind it will offer once I do.