understand DB tables in a relatively large system

What do we mean “relatively large”?
– team size 5 – 15 including system support and developers
– tables 50 – 300

tbudget for a basic understanding of all the autosys jobs?
tbudget for a basic understanding of all the tables? 20h

I see 3 unambiguous groups of tables
– R: reference tables, mostly read-only, populated from upstream
– P: product-specific, “local” tables, since we have several independent streams of products
– C: core tables shared among many products

c++ forward class declaration vs "implementation"

a FCD is the minimum declaration of a class before its use. Here are some FCD in std::iosfwd library —

template class char_traits;
class char_traits;

In C++, i see 3 levels of class declarations
1) FCD
2) class definition using method prototypes, and field compositions ie a full listing of fields. Full listing required for memory allocation.
3) class fully defined with method bodies
4?) see another post for an alternative – pure abstract classes

Usually we put #2 in *.h; client programs “#include” our class definitions by macro expansion. We seldom need to put #3 in header files, though most boost header files are #3, with important consequences for linking and compiling.

effC++ item 34 has a detailed treatment of #1 vs #2. FCD, being the minimum declaration, is also known as the “interface”, whereas #2 is known as an “implementation” and “class definition”.

Puzzled by the word “implementation”? Think of a Car as an abstract concept. Different car makers “implement” it by using concrete components. A specific implementation of car is essentially a listing of non-static fields.

Put another way, implementation means composition.

Compiler need the size of each field (possibly user-defined-types) in order to size up your Car instance. new expression and operator new calls sizeof(Car). In java, primitive fields have known sizes; all reference fields occupy 4 bytes (32-bit machine). I believe c++ compiler actually calculates and determines the address of each new’ed object — the address is not determined at runtime.

As you write a client program, you could sometimes choose to include the API classes by FCD rather than #2. I feel If you don’t open up an API object to access its members, and you only mention the class name in method signatures, then FCD suffices.

The motivation behind Item #3/4 is compile-time dependency and coupling. I feel it’s c++ specific. By the way, decoupling is one of my favorites, and is a practical priority compared to a lot of other design principles.

Google c++ coding guide gave a simple illustration of bad FCD messing up compilation, where #include is safe. In a nutshell, #include would inform compiler the inheritance relationship between B and D, which is important to compiler static binding.

shadowing a package var (perl

it’s confusing to use a lexicial $v in a sub1(), and in the same sub1() use another $v.

What kind of var is the other $v? I think it has to be a package var (both $v active simultaneously in 2 worlds), NOT another lexical. I think if it’s another lexical, then u can’t access it in sub1(). It’s shadowed and can’t be unshadowed.

If (in a legacy system) it’s a package var, then u can say $main::v. I don’t see any other way. I believe use vars qw($v) won’t work.

constructor to throw checked exception

Hi XR,
This is just a java learner’s personal opinion — i feel constructors should avoid throwing checked exceptions. Traditional if/while/switch tests are more natural , more convenient, and more maintainable than throwing exceptions. Only in a few special scenarios should a constructor throw a checked exception.
My reasons against checked exceptions thrown from a constructor? My answer is long. A constructor returns null after throwing exception. The caller method has to 1) test for null before using the new object constructed. The caller also need to 2) try-catch the exception. Of course the constructor must 3) detect the invalid input before throwing. At least 3 tests.
Whenever possible, i would move Test #3 out of constructor into the caller, so constructor won’t return null nor throw checked exceptions. How to achieve this? One standard solution is a factory. Factory does Test #3 and handles the invalid condition. Factory may return null, so the caller still need Test #1. So 2 tests to perform.
This leads to an even simpler solution. If possible, the caller method should detect invalid input (Test #3) before passing it to the constructor/factory. 1 test only.
I guess this is rudimentary. Somehow, after programming java for years, i don’t always see the simplest solution. I think there are a lot of java design patterns, java idoms, common java constructs, such as throwing checked exceptions from constructors. I learnt these and didn’t know when NOT to apply them.

to see who holds a DB lock

An experienced friend suggested “You can find the DB-loginname and the client machine connected to the DB server.” I don't
completely agree with this solution.

Very often an application loginname like “KP2_reporting” is used by 200 (realistic) applications, all by the same IT team? Many of
these applications could connect from the same production machine. At any time, up to 20 applications could be running.

I think we can also try to get the process id on the client machine, or the TCP port.

simple tibrvlisten/tibrvsend example using a winxp local rvd

Start 3 “black” DOS windows A B C.
tibrvlisten -daemon localhost:7500 -service 9012 -network “” your.subject “your msg”
tibrvsend -daemon localhost:7500 -service 9012 -network “” your.subject “your msg”

## listener and sender both complain about missing daemon.

C) Now type nothing but “rvd” to start the daemon process. It starts in 0.1 second, reporting the ip endpoint bound + admin website.

Now restart the listener and it starts listening.

Now restart sender – no more errors. Listener window shows the message.

Also, http://localhost:7580/ shows one client only — our listener

## factory functions used with STL containers

If we use “factory” to refer to any function that can create a new object, then here are some of the factory functions (including cvctor) covered in my blog or my books.

– back_inserter — always used with something_copy() functions like replace_if/replace_copy_if, count_if, remove_if/remove_copy_if … See blog
– bind2nd — used in something_if() functions like copy, remove_copy, replace_copy … See blog
– customized binary_function like “myLess”. see blog on functor-object

[19]nested STL containers #in practice

Executive summary —  favor container of smartPtr-to-container; Avoid container of raw pointer[1]. However, in many cases, the simpler container-of-container is viable and preferred.

You can also liberally use std::pair or std::tuple class templates. You can also create a simple struct of sub-containers, pairs, or simple (nonref) types, then instantiate a container template with this struct as the type argument. All of these are slightly more “natural” than container of pointers.

  • — container of container (without pointer complexity) is popular and practical:
  • One of the 3 creators of STL wrote [[STL tutorial]]. In it there are numerous examples of nested containers but very few examples of container of pointer.
  • The standard map container is kind of nested. Syntax (“map”) shows no nesting but internally, a map stores pair.
  • I used vector<vector<T>> many times, often as a matrix.
  • map<rcid, set<rcid>> p2d; // In the spreadsheet concretization code (RedMart)

[1] though many authors advocated container of raw pointers for large objects, perhaps before smart pointers. Container of raw pointer is controversial. Many authors show simple examples of them, but in practice it’s probably not so simple. Better use container of smart_ptr.

spreadsheet concretize #Junli explains why container of raw ptr is safe in this context — no q[delete] needed.

variable can’t live on the heap; only objects can

See also post [[a heap-stack dual variable]]

an object can live on the heap or the stack; but a variable can’t live on the heap. It’s either a stackVar or a field (or occasionally a global). That begs the questions

Q: what if a stack ref/ptr seated to a heap obj gets out of scope?
A: leak. unreachable object — need garbage collector.

Q: what if a field ptr/ref seated to a heap obj gets destructed with its host object?
A: the host dtor simply frees/reclaims the 4-byte memory, without calling delete() on the ptr. Item 7 [[eff c++]] says the “dtor” of a ptr is a no-op.
A: custom virtual dtor needed in the host object.

In java, a variable is either a field or a stackVar. An object is always on the heap

Segmentation Fault from sqsh

“Segmentation Fault (core dumped)” happened to my sqsh scripts many times without obvious reason. A few causes

Experiment: Put lots of commands after “/* “. These should be ignored. Run the script via sqsh -> OK. Now put a “go” at the very end -> segmentation

Experiment: remove all “/* */” comments on top, and pump into sqsh -> OK

Experiment: remove content between /* */, and pump into sqsh -> OK

Experiment: add the –, and pump into sqsh -> OK; remove the “– ” ==> segmentation fault

— $Id$

unchecked exceptions

based on [[ hardcore java ]] and http://java.sun.com/docs/books/tutorial/essential/exceptions/runtime.html

Runtime exceptions represent problems that are the result of a *programming* problem. Such problems include arithmetic exceptions, such as dividing by zero; pointer exceptions, such as trying to access an object through a null reference; and indexing exceptions, such as attempting to access an array element through an index that is too large or too small.

For example, a method can check if one of its arguments is incorrectly null. If an argument is null, the method might throw a NullPointerException.

Here’s the bottom line guideline: If a client can reasonably be expected to recover from an exception, make it a checked exception. If a client cannot do anything to recover from the exception, make it an unchecked exception.

index-only scan

“index-only” means “no data page reads”.

Before we explain index-only query, we need to explain that Most indexed queries first read the index then[2] read the data pages [1].

a typical index-only query uses a composite index [3]. Another post about “narrow index key” explains that the index key width is column width added up.

See also

http://bioivlab.ils.unc.edu/course_wikis/INLS_623/ClassWork_INLS623_Spring2008/index.php/Kyle http://www.google.com/search?q=index%2Donly+scan

[1] for structure of a page, read the db2 training manual P171
[2] follows the index leaf nodes
[3] we have no time to cover single-column index-only queries

default assignment is bitwise copy – c/c++ vs java

in C language, A = b would copy value of b into A’s memory location, resulting in 2 detached clones. Consider a C struct — the copying is bitwise.

java primitives behave the same but Java reference variables don’t. A = b would make A a “hard link” of b, so A and b point to the same object[2]. Exactly like c++ ptr copy. Note ptr A and ptr b must be compatible types. If A is a base type then slicing occurs.

Note java cloning, c/c++ assignment, c++ copier are all field by field. [2] is not field by field.

C++ treats all data types (float, char, classes…) as objects, but default assignment always works as in C (and Java primitives). Even if A and b are class types, A = b would copy value of b into A’s memory location, resulting in 2 detached clones, without creating a new object. Actually qq(A = b) may call the copier without the “explicit” modifier on the copier.

Bitwise copy means something special for a ptr field — the 4-byte content (ie address of pointee) is copied.

socket operations in perl, python and c

(“i-socket” means internet socket. Our discussion doesn't cover UD-socket ie unix domain sockets.)

1st) and easy lesson – both perl and python provide wrappers over C-level syscalls, so function names are identical across C, perl and python.

2nd) Lesson – python uses a socket object as the Dictator (remember Benevolent Dictator For Life?) whereas Perl puts a socket FILEhandle as the first argument to those functions.

Like Perl, C puts a socket FILE descriptor as first argument to those functions. However, the C file descriptor is different from a Perl file handle —
– C file descriptor is an integer either for a socket or for a disk file
– Perl file handle is internally implemented like c++ i/o streams. As compared in the post on [[pipes, streams, io classes]], all of these are built on the PIPE concept

However, both C and Perl treat a socket like a file, to support read/write.

## python features used in a web app #django

(Plone is too heavy-weight and over-kill, just like zend. Django is light-weight.)

  • MVC – model/view are classes. Controllers may/may not be classes.
  • user-defined class hierarchy
  • per-instance method-redefine
  • per-instance attribute-add/delete
  • lambda, apply — functional programming
  • try/catch – heavily used
  • decorators – Django-specific
  • introspection (basic meta-programming)

virtual op( ) overloading to replace setter method

template class Property{
Value val;

virtual void operator() (const Value& val);

* Note this is not a const method; it actually changes host object state
* it’s like a setter method, setting the this->val field
* but it uses op overloading instead of a setter method.

Q: If you have a template instance Volatility as Property<string,
double>, and then create an object myVol, how do you set its value to
1.2 ?
A: using regular setter setValue(), you would call
myVol.setValue(1.2), but using the ( ), it’s


There’s also a “getter” method.
* note the const
* note the double parenthesis.

virtual Value operator ( ) ( ) const; // note the double parenthesis.


The technique to overload q[ ( ) ] is a throwback to the constructor
initializer syntax

ConstructorName() :

Big difference now is what identifier you put in front of the paren –
put the variable name rather than the field name.

Note overloading q[ ( ) ] is the basis of many advanced techniques including STL functors, custom function objects and probably Boost bind. This overloading is needed for function-like syntax.