practical – how many ways to pass config parameters to java

-Ddir=/tmp ######## my favorite

Trailing command line args

MY_DIR=/tmp ######## env var in OS

Properties file ##### must include in class path

Spring xml file

Advertisements

AtomicBoolean is irreplaceable

People question the justification for seemingly trivial classes like AtomicSomething. Let’s take the simplest and show AtomicBoolean is irreplaceable.

compareAndSet(true, false) means “if value in main memory before this clock cycle is positive, then toggle it, otherwise do nothing. No locking please.

Read it over and over, and you would realize it’s impossible without Atomics. Volatiles can’t do it, because this involves more than a single load or store. AtomicBoolean does the load and the store in one clock cycle, using special hardware features — “cmpxchg” in intel instruction.

java’s getAndSet() is written using compareAndSet()

Q: Now, since a volatile done/stop flag is frequently used to stop a thread, shall we replace it with an AtomicBoolean?
A: no harm, but I don’t see the need.

AtomicReference + immutable = lockfree (code

import static java.lang.System.out;
import java.util.concurrent.atomic.AtomicReference;

public class AtomicAccount {
    public static void main(String[] a) {
        final AtomicAccount inst = new AtomicAccount(100);
        Runnable r = new Runnable() {
            public void run() {
                inst.addInterest();
            }
        };
        new Thread(r).start();
        new Thread(r).start();
    }

    public final AtomicReference AR = new AtomicReference();

    public AtomicAccount(float bal) {
        Account tmp = new Account();
        tmp.bal = bal;
        this.AR.set(new ImmutableAccount(tmp));
    }

    /**
     * add 1% to balance. load and store must be atomic.
     */
    void addInterest() {
        while (true) {
            ImmutableAccount b4 = this.AR.get();
            Account tmp = new Account();
            tmp.bal = b4.getBal() * 1.01f;
            ImmutableAccount update = new ImmutableAccount(tmp);
            if (this.AR.compareAndSet(b4, update)) {
                this.printBal();
                return;
            }
        }
    }

    public void printBal() {
        out.println(“current balance = ” + this.AR.get().getBal());
    }

    static private class ImmutableAccount {
        final private Account mutable = new Account();

        /**
         * if bal is a non-volatile double, we might read the 2 parts
         * un-atomically.
         *
         * @param a
         */
        public ImmutableAccount(Account a) {
            this.mutable.bal = a.bal;
            // copy other fields
        }

        public float getBal() {
            return this.mutable.bal;
        }
    }
}

class Account {
    float bal;
}

AtomicReference + immutable = lockfree

Q: what does atomic mean?
A: “within this method between first and last access to some shared object, no other thread can modify it without me knowing.”

For a mutable bean like Account, we’d love atomic operations like addInterest(). Note there’s no AtomicFloat class.

Note AtomicReference is a wrapper/bridge of the raw pointer. Somewhat similar to how a smart pointer wraps a raw pointer. Our AtomicAccount.java could be a bigger wrapper around the AtomicRefernce wrapper.

Note the cornerstone method AtomicReference.compareAndSet() works solely with addresses, in the comparison and the reseating. Much simpler and faster than equals(), esp. in a lockfree atomic context.

Q: Can we use AtomicReference to add thread safety without locking? Suppose someone supplies a mutable Account class. We will construct an AtomicAccount class meeting these requirements all without locking —

– Req: List getStocks() to atomically return all the stock symbols held. Without locking, it’s possible to see an invalid list undergoing simultaneous changes in 2 threads.
– Req: addInterest() and addFund(double). Without locking or atomic classes, you can lose updates.
– Req: getBalance()

% assumption: we can control access to Account instance. No thread should use a naked Account reference to access its state.

A key design principle — “If RAW-POINTER wrapped in the AtomicReference is not _reseated_, then the pointee object state is intact.” In other words, account is mutable only by creating a new object. —

Achieved by wrapping Account instance in an ImmutableAccount instance. Change to Account results in a new ImmutableAccount wrapping a new Account. New ImmutableAccount then replaces pointer in the AtomicReference.

Now you realize an AtomicReference is a thread-safe lookup table with just one key-value pair. Key is the unnamed default key; value is an address. Salient features:
* compareAndSet in a loop, lockfree.
* immutables
* optimistic.
* layers of wrapper[1] to give the lockfree advantage

[1] In our example, 3 wrappers on Account ) ImmutableAccount ) AtomicReference ) AtomicAccount

boost thread to run a method of my object (full source

#include
#include
class Worker{
public:
    Worker()    {
        // the thread is not-a-thread ie a dummy thread object, until we assign a real thread object
    }
    void start(int N)    {
        // pass “this” as first arg to processQueue(), which runs not as a method but as a free func in its own thread
        m_Thread = boost::thread(&Worker::processQueue, this, N);
    }
    void join(){m_Thread.join();}
    void processQueue(unsigned N)    {
        float ms = N * 1e3;
        boost::posix_time::milliseconds workTime(ms);
        std::cout << "Worker: started, will work for "
                  << ms << "ms"
                  << std::endl;
        boost::this_thread::sleep(workTime);
        std::cout << "Worker: completed" << std::endl;
    }
private:
    boost::thread m_Thread;
};
int main(int argc, char* argv[]){
    std::cout << "main: startup" << std::endl;
    Worker worker;
    worker.start(3);
    std::cout << "main: waiting for thread" << std::endl;
    worker.join();
    std::cout << "main: done" << std::endl;
    return 0;
}

Compile a boost regex/thread-based project – minGW

You configure the -L under project -> properties –> CCG -> LibraryPath

You configure the -l under project -> properties -> CCB -> settings -> toolSettings tab -> MingwC++Linker -> library
— regex
cd C:xbakideWorkspaceec_boost1Debug
g++ -IC:boost_1_44_0 -O0 -g3 -Wall -c -fmessage-length=0 -osrcec_boost1.o ..srcec_boost1.cpp
g++ -LC:boost_1_44_0stagelib -oec_boost1.exe srcec_boost1.o -lboost_regex-mgw44-1_44

— boost::thread is more complicated cos the lib file(s) you built with bjam seem to be a hybrid of static and dynamic lib. Perhaps a static import library used to pull in the dll file (yes there is a dll produced). In that case, at runtime the dll must be available on PATH.

cd C:xbakideWorkspaceec_boost1Debug
g++ -IC:boost_1_44_0 -O0 -g3 -Wall -c -fmessage-length=0 -osrcec_boost1.o ..srcec_boost1.cpp
g++ -LC:boost_1_44_0stagelib -Bstatic -oec_boost1.exe srcec_boost1.o -lboost_thread-mgw44-mt-1_44
C:xbakideWorkspaceec_boost1Debugec_boost1.exe

If you see error about missing dll, then copy the named dll to the location of your exe, or a %PATH% dir. Why the error? Dependency Walker shows LIBBOOST_THREAD-MGW44-MT-1_44.DLL is a dependency of your new born exe. Apparently, there’s no real static linking of the boost thread lib.

##decouple to refactor a degenerating codebase

An OO design often starts cleanly decoupled, then gets /entangled/. Over time, tight coupling invariably creeps in (among other code smells) –>

* a class having (too) many fields pointing to other classes we wrote. The laziest way to put some existing logic into another class is to add a field , so-called “dependency” or “service”.
** too many setters
** complex constructors
* too many “new SomeClass” all over the place
* too many objects passed around as method arguments
* too many “public” members

Drawback — rigid, fragile — see post on …

— Some of my favorite refactoring fixes —
– a lot of assertions to document my refactoring assumptions.
– c# style static classes — all-static stateless classes
– black-box Facade
– More shared interfaces
– Factory
– Tuple classes for many-to-many association.
– POJO/DTO classes without methods — you could slowly add one or 2 methods but not recommended when cleaning up a degenerate codebase.
– Enums

– (code smell) use exceptions to break out of loops and call stacks.

return value optimization

I got a surprise when I run the program below with g++ (mingw port).

* returnLocalVar() should return by value, by calling the copy constructor, but copy constructor didn't get called.
* localCat object should be destructed when it goes out of scope at end of the function, but no it was kept alive till end of main()
* ret is a local variable in main(), not a reference, so i expect ret to be a copy of localCat, with a separate address, but no they 2 variables have the same address.

#include
#define S(x) std::cout<<#x" address =\t"<<&x<<"\n";
using namespace std;
struct CAT {
    CAT() {
        cout < no-arg constructor with addr = ” << this << "\n";
    } // default constructor
    ~CAT() {
        cout << this << " Destructor called!\n";
    }
    CAT(const CAT& rhs) {
        cout < copy constructor ” << this << " <- " << &rhs << "\n";
    }
};
CAT returnLocalVar() {
    CAT localCat;
    cout << "leaving returnCat\n";
    return localCat;
    // localCat is not destructed if result is assigned
}
int main() {
    CAT ret = returnLocalVar();
    S(ret);
    cout << "done!\n";
}

Answer from a friend:

return value optimization (RVO). It is an optimization specifically blessed by the C++ specification. While the compiler is creating the returnLocalVar function, it is allowed for it to create the CAT temporary in main’s stack space instead of returnLocalVar’s stack space. Then, instead of returning it by making a copy, it just leaves it there on the stack where main can find it. The compiler can’t always figure out whether that would be safe, so it only does it in predictable circumstances.

Usage of ScheduledFuture and Callable in our pricing engine

The dependency manager may have submitted 1000 requests for repricing and the engine has gotten through 300 of them. If the dependency manager knows that for example position 106 no longer needs a revaluation, it can use the future object to cancel it.

Future also gives us access to the result of the task, but we mostly use Future to cancel tasks for multithreading efficiency

release test failures and code quality

(blogging again) XR, You mentioned some colleagues often release (to QA) code that can’t complete an end-to-end test .. (Did I got you wrong?)

That brings to mind the common observation “95% of your code and your time is handling the unexpected”. That’s nice advertisement and not reality — a lot of my GS colleagues aggressively cut corners on that effort (95% -> 50%), because they “know” what unexpected cases won’t happen. They frequently assume (correctly) a lot of things about the input. When some unexpected things happen, our manager can always blame the upstream or the user so we aren’t at fault. Such a culture rewards cutting-corners.

I used to handle every weird input combination, including nulls in all the weird places. I took the inspiration from library writers (spring, or tomcat…) since they can’t assume anything about their input.

I used to pride myself on my code “quality” before the rude awakening – such quality is unwanted luxury in my GS team. I admit that we are paid to serve the business. Business don’t want to pay for extra quality. Extra quality is like the loads of expensive features sold to a customer who just needs the basic product.

As we discussed, re-factoring is quality improvement at a price and at a risk. Business (and manager) may not want to pay that price, and don’t want to take unnecessary risk. I often do it anyway, in my own time.