system instrumentation – windows is murkier than linux

See also

http://bigblog.tanbin.com/2013/07/untold-horror-stories-about-microsoft.html

http://bigblog.tanbin.com/2013/12/fwd-pure-language-complexity-vs-tool.html

I recently browsed [[inside windows debugging]]. In the past I also browsed a few *nix system programming books. I'm more afraid of windows than *nix. Windows is more dark and murky; linux is more open and consistent. As a beginner, I feel in linux the challenge is interpreting tool output but on windows it's getting past the installation and security restrictions.

* complication: GUI — As explained in this blog (http://bigblog.tanbin.com/2013/04/for-administration-prefer-command-line.html), GUI tends to hide a lot of features in layers and compartments. Console tools provide the same amount of power features …

** if there's a security or registry or other issues that reduce the provided feature set to below 100%, a console tool is more likely to report it to the user but GUI tools often “intelligently” handle it as if it's normal.

MSVS is the primary windows instrumentation tool – very complicated.

* complication: tool installation – Linux tools are not easy to install, but Windows tools tend to be even harder to install.

** registry.

** On any platform GUI tools are, on average, harder to install. Linux tools are less likely GUI.

* complication: security restrictions – for a novice, 1/3 of the problems with windows tools are related to security/permission. Even under an administrator account, there are many security restrictions we may need to painstakingly disable! I don't have any example now. I feel windows was criticized as vulnerable so Microsoft added a lot of security features.

** some security features seem to be controlled by the domain administrators.

* complication: environment factors – If you run the same tool in 2 machines with different results it's often due to environment factors. In Linux, environment variables and some utilities are the only environment factors I know. In windows there seem to be many more places to check.

** registry

** certificates

** control panel items

** secpol.msc

* complication: tool availability – many windows tools are commercial. On windows, I generally find the open-source tools better-quality, but for some tasks there's only one tool – shipped by Microsoft. If I feel it is often crippled. On linux the best tools are invariably open-source.

* complication: undocumented features — Even if you follow the all documented procedures, still you may hit something unexpected, due to undocumented features. This happens more often on windows then linux.

Note undocumented features affect not only instrumentation tools, but other components of the platform, which affect our debugging.

Fwd: pure language complexity ^ tool complexity

(“Tool” includes the entire system used in the dev process.)

YH,

You asked me why my OC manager likes to beat me up. I thought about it. Here’s one reason.

My manager complains I’m slow with my development projects. In this job (as in many other jobs), people don’t say my code quality or design is bad. I think most of the code they write isn’t great either. We are just hoping to churn out working code on time. I do hope to but won’t try too hard to make my code beautiful, low-maintenance, testable, easy to change, easy to understand. Implicitly, that’s the unspoken agreement with the boss. One quality where I exceed minimum requirement is error-condition handling — i want my code to behave reasonably under any conceivable condition. I don’t want “undefined behavior”. I don’t want surprises. Anyway, my boss only complains about how slow I complete projects, not about quality. Invariably it boils down to troubleshooting skill like googling, asking around, understanding logs, and diagnostic tools. Sometimes not easy for anyone. Most of these technical problems involve more than code complexity in my c# codebase, and now I realize that’s the key.

Now I believe my c# language skill is perhaps better than the average c# developer. For eg, I’m confident taking on technical challenges involving threading, linq, complex data structures, complex delegates+events. (The toughest coding interviews on the west coast sometimes cover nothing but a little tight/dense algorithm. These can be extremely IQ-intensive, more than I can handle but I feel comfortable that I can beat most c# developers at this game.) In a nutshell, if the problem is purely about c# language, I am confident. Most real world technical problem-solving goes beyond that. There are too many of these problems to enumerate. Below are just a few examples.

eg: When the same code works in one environment not in another environment, I feel 30% of the time it’s due to some permission or security issue. Error message is usually misleading. Even worse when nothing happens, without any error msg. Can take ages to uncover the root cause.

eg: GUI troubleshooting always involves some tool and some infrastructure that’s not written by me…

eg: GUI system is invariably a client to some server-side. There’s always some networking involved.

eg: Async communication might be considered a pure language complexity but in reality troubleshooting often involves some vendor-supplied infrastructure code which sets up the communication and threading.

eg: WCF and remoting always involves tools to generate proxy, network layer, serialization…

eg: My biggest source of troubleshooting headache is VisualStudio… I had many problems with debugger, with build, with project references…

In order to reproduce a problem, I often need to check out and build many projects, configure many modules, hunt down all the required libraries, set up all the dependencies, install a bunch of infrastructure software… The set-up can take days for the uninitialized, and is error-prone and not fully documented. This happens everywhere so I’m not complaining, but this often leads to a lot of delay and complaint from boss.

factory returning a (smart) ptr

Background — This is one example of “too many variations”. A simple idea become popular. People start mix-n-match it with other ideas and create many, many interesting questions. In practice we should stick to a few best practices and avoid the hundreds of other variations.

I think it’s quite common (see [[safe c++]] P51) to have a factory function creating a heap “resource”. Given it’s on heap, factory must return the object by pointer. Easy to hit memory leak. Standard solution is return by ref count smart ptr.

Q: what if I forget to destruct the obj returned?
A: I feel it’s uncommon and unnatural. (More common to forget deletion.) Often the object returned i.e. the smart pointer object is on the stack – no way to “forget”.

Note if a raw pointer variable is on the stack, then the stack unwinding will deallocate/reclaim the memory, but won’t call delete on it!

Q: what if the object returned is saved as a field like myUmbrella.resource1
A: I feel this is fine because the myUmbrella object will be destructed by someone’s responsibility. That would automatically destruct the smart pointer field this->resource1.

Alternative to the ref count smart ptr, we can also return a scoped pointer. See [[safe c++]].

Q: what if factory need to return a pointer to stack?
A: I don’t think so. Pointee goes out of scope….
A: I feel most “resources” are represented by a heapy thingy. Alternatively, it’s also conceivable to hold a “permanent” resource like globally. Memory management is non-issue.

By the way, [[safe c++]] also mentions the __common__ pattern of a (non-factory) function returning a pointer to an existing object, rather than a “new SomeClass”. The challenges and solutions are different.

[[inside windows debugging]]

P38 has a half-pager comparison of the 2 main debuggers

^ MSVS – dev environment, with source code available

^ windbg – production environment. For post-coding, without source code.

** script/SQL debugging – only MSVS

P38 points out the free MSVS-express lacks certain debugging features. WinDBG is Completely free.

Q: does “windows debugger” mean windbg + minor tools?

–symbol files, symbol servers, *.pdb files

P53

P54 – “public symbols” and the microsoft online public symbol server.

replicate eq-fwd contract, assuming a single dividend

See also
http://www.matthiasthul.com/joomla/attachments/article/121/ForwardReplication.pdf

Note replication portfolio is always purchased as a bundle, sometime (time t) before expiry (denoted time T).

First, let’s review how to replicate a forward contract in the absence of dividends. The replication portfolio is {long 1 share, short K discount bonds}. To verify, at T the portfolio payout is exactly like long forward. By arbitrage argument, any time before expiry the portfolio value must at all times equal the fwd contract’s price. I will spare you the math formula, since the real key behind the math is the replication and arbitrage.

Now, suppose there’s a percentage dividend D paid out at time Td before T. In this case, let’s assume the dividend rate D is announced in advance. To reduce the abstractness, let’s assume D=2%, K=$100, the stock is IBM. We are going to reinvest the dividend, not use it to offset the purchase price $100. (This strategy helps us price options on IBM.)

The initial replication portfolio now adjusts to –{ long 0.98 IBM, short 100 discount bonds}. At T, the portfolio is exactly like long 1 forward contract. Please verify!

(In practice, dividends are declared as fixed amount like $0.033 per share whatever the stock price, but presumably an analyst could forecast 2%.)

In simple quant models, there’s a further simplification i.e. continuous dividend yield q (like 2% annually). Therefore reinvesting over a period A (like 1Y), 1 share becomes exp(qA) shares, like exp(0.02*1) = 1.0202 shares.

Q: delta of such a fwd contract’s pre-maturity value? Math is simple given a good grip on fwd contract replication.
A: rep portfolio is { +1 S*exp(-qT),     -K bonds }.
A: key concept — the number of shares (not share price) in the portfolio “multiplies” (like rabbits)  at a continuous compound rate of q. Think of q = 0.02.
A: In other words

   F0 = S0*exp(-qT) – K*Z0

Differentiating wrt S0, delta = exp(-qT), which degenerates to 1 when q=0.