sys instrumentation: win32 murkier than linux #c++?

See also

http://bigblog.tanbin.com/2013/07/untold-horror-stories-about-microsoft.html

http://bigblog.tanbin.com/2013/12/fwd-pure-language-complexity-vs-tool.html

I recently browsed [[inside windows debugging]]. In the past I also browsed a few *nix system programming books. I'm more afraid of windows than *nix. Windows is more dark and murky; linux is more open and consistent. As a beginner, I feel in linux the challenge is interpreting tool output but on windows it's getting past the installation and security restrictions.

* complication: GUI — As explained in this blog (http://bigblog.tanbin.com/2013/04/for-administration-prefer-command-line.html), GUI tends to hide a lot of features in layers and compartments. Console tools provide the same amount of power features …

** if there's a security or registry or other issues that reduce the provided feature set to below 100%, a console tool is more likely to report it to the user but GUI tools often “intelligently” handle it as if it's normal.

MSVS is the primary windows instrumentation tool – very complicated.

* complication: tool installation – Linux tools are not easy to install, but Windows tools tend to be even harder to install.

** registry.

** On any platform GUI tools are, on average, harder to install. Linux tools are less likely GUI.

* complication: security restrictions – for a novice, 1/3 of the problems with windows tools are related to security/permission. Even under an administrator account, there are many security restrictions we may need to painstakingly disable! I don't have any example now. I feel windows was criticized as vulnerable so Microsoft added a lot of security features.

** some security features seem to be controlled by the domain administrators.

* complication: environment factors – If you run the same tool in 2 machines with different results it's often due to environment factors. In Linux, environment variables and some utilities are the only environment factors I know. In windows there seem to be many more places to check.

** registry

** certificates

** control panel items

** secpol.msc

* complication: tool availability – many windows tools are commercial. On windows, I generally find the open-source tools better-quality, but for some tasks there's only one tool – shipped by Microsoft. If I feel it is often crippled. On linux the best tools are invariably open-source.

* complication: undocumented features — Even if you follow the all documented procedures, still you may hit something unexpected, due to undocumented features. This happens more often on windows then linux.

Note undocumented features affect not only instrumentation tools, but other components of the platform, which affect our debugging.

Advertisements

Fwd: pure language complexity ^ tool complexity

(“Tool” includes the entire system used in the dev process.)

YH,

You asked me why my OC manager likes to beat me up. I thought about it. Here’s one reason.

My manager complains I’m slow with my development projects. In this job (as in many other jobs), people don’t say my code quality or design is bad. I think most of the code they write isn’t great either. We are just hoping to churn out working code on time. I do hope to but won’t try too hard to make my code beautiful, low-maintenance, testable, easy to change, easy to understand. Implicitly, that’s the unspoken agreement with the boss. One quality where I exceed minimum requirement is error-condition handling — i want my code to behave reasonably under any conceivable condition. I don’t want “undefined behavior”. I don’t want surprises. Anyway, my boss only complains about how slow I complete projects, not about quality. Invariably it boils down to troubleshooting skill like googling, asking around, understanding logs, and diagnostic tools. Sometimes not easy for anyone. Most of these technical problems involve more than code complexity in my c# codebase, and now I realize that’s the key.

Now I believe my c# language skill is perhaps better than the average c# developer. For eg, I’m confident taking on technical challenges involving threading, linq, complex data structures, complex delegates+events. (The toughest coding interviews on the west coast sometimes cover nothing but a little tight/dense algorithm. These can be extremely IQ-intensive, more than I can handle but I feel comfortable that I can beat most c# developers at this game.) In a nutshell, if the problem is purely about c# language, I am confident. Most real world technical problem-solving goes beyond that. There are too many of these problems to enumerate. Below are just a few examples.

eg: When the same code works in one environment not in another environment, I feel 30% of the time it’s due to some permission or security issue. Error message is usually misleading. Even worse when nothing happens, without any error msg. Can take ages to uncover the root cause.

eg: GUI troubleshooting always involves some tool and some infrastructure that’s not written by me…

eg: GUI system is invariably a client to some server-side. There’s always some networking involved.

eg: Async communication might be considered a pure language complexity but in reality troubleshooting often involves some vendor-supplied infrastructure code which sets up the communication and threading.

eg: WCF and remoting always involves tools to generate proxy, network layer, serialization…

eg: My biggest source of troubleshooting headache is VisualStudio… I had many problems with debugger, with build, with project references…

In order to reproduce a problem, I often need to check out and build many projects, configure many modules, hunt down all the required libraries, set up all the dependencies, install a bunch of infrastructure software… The set-up can take days for the uninitialized, and is error-prone and not fully documented. This happens everywhere so I’m not complaining, but this often leads to a lot of delay and complaint from boss.

factory returning a (smart) ptr

Background — This is one example of “too many variations”. A simple idea become popular. People start mix-n-match it with other ideas and create many, many interesting questions. In practice we should stick to a few best practices and avoid the hundreds of other variations.

I think it’s quite common (see [[safe c++]] P51) to have a factory function creating a heap “resource”. Given it’s on heap, factory must return the object by pointer. Easy to hit memory leak. Standard solution is return by ref count smart ptr.

Q: what if I forget to destruct the obj returned?
A: I feel it’s uncommon and unnatural. (More common to forget deletion.) Often the object returned i.e. the smart pointer object is on the stack – no way to “forget”.

Note if a raw pointer variable is on the stack, then the stack unwinding will deallocate/reclaim the memory, but won’t call delete on it!

Q: what if the object returned is saved as a field like myUmbrella.resource1
A: I feel this is fine because the myUmbrella object will be destructed by someone’s responsibility. That would automatically destruct the smart pointer field this->resource1.

Alternative to the ref count smart ptr, we can also return a scoped pointer. See [[safe c++]].

Q: what if factory need to return a pointer to stack?
A: I don’t think so. Pointee goes out of scope….
A: I feel most “resources” are represented by a heapy thingy. Alternatively, it’s also conceivable to hold a “permanent” resource like globally. Memory management is non-issue.

By the way, [[safe c++]] also mentions the __common__ pattern of a (non-factory) function returning a pointer to an existing object, rather than a “new SomeClass”. The challenges and solutions are different.

[[inside windows debugging]]

P38 has a half-pager comparison of the 2 main debuggers

^ MSVS – dev environment, with source code available

^ windbg – production environment. For post-coding, without source code.

** script/SQL debugging – only MSVS

P38 points out the free MSVS-express lacks certain debugging features. WinDBG is Completely free.

Q: does “windows debugger” mean windbg + minor tools?

–symbol files, symbol servers, *.pdb files

P53

P54 – “public symbols” and the microsoft online public symbol server.

replicate eq-fwd contract, assuming a single dividend

See also
http://www.matthiasthul.com/joomla/attachments/article/121/ForwardReplication.pdf

Note replication portfolio is always purchased as a bundle, sometime (time t) before expiry (denoted time T).

First, let’s review how to replicate a forward contract in the absence of dividends. The replication portfolio is {long 1 share, short K discount bonds}. To verify, at T the portfolio payout is exactly like long forward. By arbitrage argument, any time before expiry the portfolio value must at all times equal the fwd contract’s price. I will spare you the math formula, since the real key behind the math is the replication and arbitrage.

Now, suppose there’s a percentage dividend D paid out at time Td before T. In this case, let’s assume the dividend rate D is announced in advance. To reduce the abstractness, let’s assume D=2%, K=$100, the stock is IBM. We are going to reinvest the dividend, not use it to offset the purchase price $100. (This strategy helps us price options on IBM.)

The initial replication portfolio now adjusts to –{ long 0.98 IBM, short 100 discount bonds}. At T, the portfolio is exactly like long 1 forward contract. Please verify!

(In practice, dividends are declared as fixed amount like $0.033 per share whatever the stock price, but presumably an analyst could forecast 2%.)

In simple quant models, there’s a further simplification i.e. continuous dividend yield q (like 2% annually). Therefore reinvesting over a period A (like 1Y), 1 share becomes exp(qA) shares, like exp(0.02*1) = 1.0202 shares.

Q: delta of such a fwd contract’s pre-maturity value? Math is simple given a good grip on fwd contract replication.
A: rep portfolio is { +1 S*exp(-qT),     -K bonds }.
A: key concept — the number of shares (not share price) in the portfolio “multiplies” (like rabbits)  at a continuous compound rate of q. Think of q = 0.02.
A: In other words

   F0 = S0*exp(-qT) – K*Z0

Differentiating wrt S0, delta = exp(-qT), which degenerates to 1 when q=0.

barebones RAII smart ptr classes

If RAII and memory management is the main purpose, I feel a better word for the class template is a “capsule class”. I feel the boost implementations are too complicated and hard to modify. Here are some small but realistic implementations

[[c++FAQ]] has a one-pager RAII capsule class.

[[safe c++]] has a 2-page-short source code for a real-world capsule. Includes both ref count and scoped versions.

http://www.artima.com/cppsource/bigtwo.html introduces a very simple RAII capsule. One of the simplest yet useful smart pointers. I think the entire class needs nothing but … big3 + ctor.

A few observations —

* It’s not reliable enough if the capsule ctor can throw. For RAII to provide the ironclad guarantee, the ctor of the “capsule” class (smart ptr or not) should be simple and robust, and no-throw. If the capsule is not even locked during construction, then it offers no protection no guarantee.

* (Smart pointer always needs a copy policy) It’s meaningless unless every client “goes through” the wrapper to get to the raw pointer. Raw pointer direct access is dangerous and should be completely avoided.

* It’s meaningless without the exception risk. RAII is mostly for exceptions and programmer forgetfulness.
* It’s meaningless if there’s no need for an ironclad “guarantee“.
* It’s meaningless if there’s no need for “automatic” cleanup.
* It relies on the dtor feature. RAII relies on dtor. Smart pointer relies on dtor.
* It relies on local stack variables,but …
* It’s unusable unless the original pointer is for a heap object.
… These keywords are the heart of RAII idiom.

Given the heavy reliance on dtor, the dtor should be fairly simple. I don’t think it should be virtual.

##windows GTD skillset, my take

(master copy? Recrec)

Dotnet is like java, right? Dotnet developers operate in a sandbox similar to the java developer?

Wrong! I feel a windows developer team operates in an ecosystem that’s fundamentally different from a java development ecosystem. Differences lie beneath the surface.

Since all of my java servers run in linux (or unix), I also need some linux platform knowledge, but nothing in-depth.

A windows developer needs a lot of specific (platform) skills. The list below is what I feel I need to pick up. [[Inside windows debugging]] by a MS veteran points out loads of essential know-how required for dotnet (or pre-dotnet) hacking beyond the MSVS debugger. From this book i get the sense that every serious windows developer need (non-trivial) windows know-how.

As an analogy, serious sql coders need (non-trivial) knowledge of optimizer and query plan.

I feel the average windows developer don’t have the capacity to know all of these. Perhaps some guys specialize in winforms some in wpf, some specialize in web development, some specialize in VB, some specialize in c# server side… Yet it’s not inconceivable for some lead developer/architect to have some experience with all of these, and some deep experience in a few topics.

* windows TCP/IP – Microsoft has its own implementation
* windows threading – many small differences from java threading, even though the concepts match.
* windows web service (SOAP etc) – Microsoft has its own implementation
* debugger – remote debugger
* xml parsing – Microsoft has its own implementation
* office integration (COM), excel addin
* IDE – many many must-know 
* project configuration, 
* application configuration
* GUI dev – silverlight
* GUI dev – wpf
* set up WCF service and host
* c# loggers
* powershell, DOS scripting
* MSSQL

———
* iis
* messaging server
* sharepoint server 
* exchange server 
* GUI dev – winforms
* GUI dev – MFC
* web development – silverlight
* web development – ASP, ASP.net
* web development – vbscript, jscript etc

The blog title says “windows” because I believe a serious dotnet developer always knows windows well beyond dotnet. Many __dotnet__ dev skills require windows platform knowledge. (In java projects platform knowledge is useful but not as essential as in dotnet.) I believe a dotnet developer has to become a windows developer to be truly competent.

– COM integration
– MS-Office integration
– interop with VB, VC, both rich-history technologies retrofit into dotnet
– windows debuggers
– sysinternals tools
– registry
– application installer/uninstaller
– event viewer
– performance counters
– dotnet threading is based on win32
– WCF is based on a earlier Microsoft remoting technologies

java spring IV questions from friends #YH etc

Q: How many ways of instantiating a Spring container?

Q: How many ways of injecting a bean in Spring XML config

Q: What's drawbacks of using constructor as injection mean? circular reference, what's exception will be thrown?

Q: Spring annotation. If a bean must be provided with a constructor with another bean injected, what's the attribute of the annotation should be used to enforce it.

Q: What're the scopes in a bean tag in Spring XML?

Q: If a scope being prototype, what will return from using “getBean()”