opaque challenge:(intermediate)data browser #push the limit

Opacity — is a top 3 (possibly biggest) fear and burden in terms of figure-things-out-relative-to-cowokers on a localSys. The most effective and direct solution is some form of instrumentation tool for intermediate data. If you develop or master an effective browsing tool, however primitive, it would likely become a competitive advantage in terms of figure-out speed, and consequently earn you some respect.

LocalSys — I feel most of the effective data browser knowledge is localSys knowledge.

If you are serious about your figure-out speed weakness, if you are seriously affected by the opaque issues, then consider investing more time in these browsing tools.

Hard work, but worthwhile.

  • eg: Piroz built a Gemfire data browser and it became crucial in the Neoreo project
  • #1 eg: in my GS projects, the intermediate data was often written into RDBMS. Also important — input and output data are also written into RDBMS tables. Crucial in everyday trouble-shooting. I rank this as #1 in terms of complexity and value. Also this is my personal first-hand experience
  • #2 eg: RTS rebus — during development, I captured lots of output CTF messages as intermediate data… extremely valuable
    • Once deployed, QA team relied on some web tools. I didn’t need to debug production issues.
    • I remember the static data team save their static data to RDBMS, so they relied on the RDBMS query tool on a daily basis.

Now some negative experiences

  • eg: I’m not too sure, but during the Stirt QZ dev projects I didn’t develop enough investigation skill esp. in terms of checking intermediate data.
  • eg: in Mvea, we rely on net-admin commands to query order state, flow-element state and specific fills… Not as convenient as a data store. I never became proficient.
    • I would say the FIX messages are logged consistently and serve as input and output data browser.

Many projects in my recent past have no such data store. I don’t know if there’s some effective solution to the opacity, but everyone else face the same issue.

Advertisements

@outage: localSys know-how beats generic expertise

When a production system fails to work, do you contact

  • XX) the original developer or the current maintainer over last 3Y (or longer) with a no-name college degree + working knowledge of the programming language, or
  • YY) the recently hired (1Y history) expert in the programming language with PhD and published papers

Clearly we trust XX more. She knows the localSys and likely has seen something similar.

Exception — what if YY has a power tool like a remote debugger? I think YY may gain fresh insight that XX is lacking.

XX may be poor at explaining the system design. YY may be a great presenter without low-level and hands-on know-how.

If you discuss the language and underlying technologies with XX he may show very limited knowledge… Remember Andrew Yap and Viswa of RTS?

Q: Who would earn the respect of teammates, mgr and external teams?

XX may have a hard time getting a job elsewhere .. I have met many people like XX.

camp_out]ff: %%coping strategy/strength

Exec summary: If I find myself engaged, I would want to spend extra quiet hours including weekends working in office and make progress on localSys. I would be able to apply myself and match my brain power against the accumulated brainpower baked into the local code base.

By “Camp-out” I mean all forms of extra time spent working in office. Camp-out is a concrete form of engagement. Usually the engaging period is a honeymoon but hopefully it can become a more sustained focus on localSys. If it does happen (I won’t push myself too hard…) I would hit better GTD, productivity, and earn some respect from some colleagues (not necessarily the boss).

More importantly, I would feel proud of myself and free of regret or guilt. See warning below.

Capture the mood. Eg: You get an engaging project that give you insight into portable GTD, tools, or localSys.

  • Xp: Qz — I did capture the mood a few (like 3+) times. I felt really good even though the camp-out didn’t “save me” in the end. Like U.S. presidential election, the “end” result depends on many many things so it’s unwise to use it as a big paint brush and wipe out all the effort.
  • Xp: Macq — I made big efforts to camp out to work on c++ build system and python. No regrets. The camp-out didn’t really build my foundation in localsys as the expectation was too high. No regrets!
  • xp: OC — I did a few times, mostly to learn c#. No regrets.

— Warning: camp-out may not earn me respect, GTD, enough localSys mileage. Ashish feels I would survive.

I feel I may still get PIP, but after such an all-out effort, I would be decisive to leave. Consider Qz and Macq.

— I see this as my competitive strength and an effective coping strategy.

I used to feel I’m kind of inadequate if I have to camp out and sacrifice family time just to keep my head above water. Wrong! Most of my peers simply don’t have the absorbency, and productive solitude to do this. Many can’t even imagine it.

I said my son may have more or less math abilities, but his effort (chiefly absorbency) is probably lower than his classmates. My daughter’s abilities is unknown, but her absorbency is better than her brother. Similarly, my absorbency is higher than my peers as demonstrated by camp-out.

— Initially, May need 6M-9M to get over the initial hump. I can even rent a temp home for 6M if I feel engaged. See kids only on weekends.

— As contractor I feel the motivation but zero pressure to camp out; As employee I feel pressure to camp out. Clearly I feel better as contractor.

— Q: In my experience, part of the camp-out hours are often spent on blogging (flight control) and morale-boosting workout. OK?

— Q: However, if code is too opaque, I won’t be able to figure things out 😦 What can I do?

— Q: What if I receive a PIP.. will I be able to persuade myself to camp out?
A: Yes. I did in GS, Qz and Macq. No honeymoon, but I have been able to overcome the huge negative energy, convert it partially to motivation and get myself to camp out and focus on GTD.

I like this imagery — I felt like a boxer on my back-foot, not hopeful but not pessimistic either.

— Q: If in the mood for localSys but need to sleep? Shall i go home?
A: don’t go home to sleep. Sleep in office. Go home to eat then come back to office.

 

[17]predict 2-5Y job satisfaction #OC surprise

Q: did the learning actually help with my job interviews (am not salary-focused like Mithun)? This is a key feedback.
A: Yes to some extent

The “table” is subject to frequent change, so I keep it in recoll (was a google sheet). Here some notes:

  • stigma/appreciation/respect(zbs) — turns to be the #1 key to job satisfaction, but appreciation is tricky. Bonus can break it, performance review can break it, other things can break it too. I often felt like “damaged goods”.
    • In Mac and Stirt (and OC too), managers considered me good enough for transfer to other teams.
    • retrospective can be more negative than dissatisfaction then-n-there, for Macq, Stirt
  • salary + other income — turns out to be insignificant when I have inner confidence and self-esteem. It is still one of the most important factors. When it’s very high, it overrides almost everything else.
  • distractions — do hurt my O2 for GTD, zbs development and self-learning
  • traction — positive feedback, includes zbs growth, instrumentation, self confidence, IV, catching up or surpassing peers
  • strategic orgro/stagnation — turns out to be a white elephant.
  • Many of the top factors are “perceptions” rather than “hardships”
    • perceptions — self-esteem@comp; strategic tsn; engaging… #best eg@perception — peer comparison
    • hardships — mkt depth (job pool size); workload; family time; commute; salary

! OC job was actually not bad if there were some appreciation from boss. However, the new, strategic specialization didn’t bring enough satisfaction.

! Verizon job experience was rather positive. I was on the rise, in my prime. It all ended when I moved to GS. I should have quit GS earlier. Citi was the start of another prime period. Prime mostly in terms of self-confidence, self-esteem …

My prediction — to have a satisfying (not necessarily strategic) job next time,

  • I need the #1 factor — appreciation.
  • A well-paid java job will mostly likely make me feel good.
  • LearningSomethingNew and engagement will NOT be a deciding factor (Recall c#/py experiences). I will still make time to learn something, just like in 95G

 

##family safety enhancers: next sg job#respect[def]

see predict next 3-5Y job satisfaction #engaging

see what determined7past job satisfaction #nextSgJob

Conclusion: As of Apr 2018 the top 2 enhancers are

  1. respect — more than “thank-you-for-trying” appreciation
  2. MktDepth — market depth
  3. –others…
  4. comp? important to satisfaction but not really important to “safety”
  5. engagement? important to satisfaction then-n-there but not really important to safety

##[18]realistic 2-10Y career planning guidelines

Background: not easy to have a solid plan that survives more than 3Y. Instead of a detailed plan, I will try to manage using a few guidelines.

  • –top 5 “guidelines” [1]
  • respect/appreciation/appraisal(esp. by manager) — PIP/stigma/trauma/damagagedGood. Let’s accept: may not be easy to get
  • Singapore — much fewer choices. Better consider market-depth^elite domain
  • Expertise accu or sustained focus — holy grail
  • ——– secondary:
  • family time — how2get more family time #a top3 priority4Sg job
  • trySomethingNew — may/not be justifiable
    • stagnation — could be the norm
    • engaging — keep yourself engaged, challenged, learning, despite the stagnation
  • interviews — Let’s accept : extremely important to me but much harder in Singapore. Even in the U.S. I may need to cut down.
  • distractions — Let’s accept
  • Peer pressures — Let’s accept.
  • Entry-barrier — could be too high for me in the popular domains like algo trading
  • non-lead dev role — Let’s embrace. Don’t feel you must move out or move up. Hands-on coding is gr8 for me. Feel good about it
  • Shrinking Choices — many employers implicitly prefer younger programmers
  • Entry-barrier — could be too low for some young guys — the popular domains will have many young guys breaking in
  • Churn — Avoid

[1] I didn’t say “priorities”

 

[19]C#job #eg: Ashish

See also enough c# mileage accummulated@@ and exposure: semi-automatic(shallow)Accu #$valuable contexx

As I told XR, a C# job offers opportunity for steeper growth curve, and higher chance of learning something useful [1]. It could provide another boost to my c++/java zbs.

As I told XR, some c++ shops are on VC++, and I am rather weak on windows and MSVS.

[1] In contrast, a regular c++ (and esp. java) job might hit a plateau of slow growth and diminishing return.

Therefore, c# job sounds more “strategic”. Strategic is not about long-term career development, but about engaging. With something “strategic” I might be more engaged, but I might lose the interest in X months too.

–Ashish’s team specifically

  • 🙂 Good chance of appreciation. Ashish isn’t as strong as Kevin.
  • 🙂 smaller codebase
  • 🙂 bitcoin trec (+shallow dnlg) will be a new addition to my /repertoire/ on my profile
  • 🙂 no risk of reputation damage at a big bank

##respect,$,familyTime,spareTime..all benefit from强项

If I take a 强项 job, I kind of sacrifice my muscle building and TrySomethingNew, so I better get good money …, but now I feel I don’t have to.

A typical 强项 job would use (the most complete list)

  • math, analytics
  • high volume data processing
  • unix wizardry
  • heavy text processing
  • heavy SQL
  • heavy scripting
  • some http programming
  • data analysis, perhaps using SQL, scripting etc
  • some combo of java/c++/c#/swing
  • threading

##criteria: domains2specialize over30Y #respect

See also post on top 5 expertise I could teach.

In the US job market, people often ask “What do you specialize in?”. I think most non-managers in this industry, esp. the successful ones, do specialize in something. Whether you like it or not, you are often perceived that way.

Clearly, many professionals are jack of many trades (or a jack of few trades), and don’t have any real expertise, depth or insight. Depending on your view, this may not be a problem for them.

Like property evaluation, I have a list of criteria:

  1. theoretical complexity — so most peers can’t master it. I get lower stress. For example, Threading, statistics, pricing models, algorithms, data science? …
  2. market depth (entry barrier) — eg: quant is too hard to get into and very few jobs in the mid-range or low-end
  3. opportunities in research/teaching, for my later years. Relatively few choices.
  4. aptitude — (aka personal advantage) easier to add value and receive appreciation and recognition.
  5. Something I believe in or care about, such as personal investment, or health. Self-knowledge: If it has commercial value I will care about it.
  6. ———– Rest are secondary —————-
  7. pathway to self-employment
  8. (obvious) accumulation and low churn — Look at grandpa
  9. premium on job market — low priority in my later career
  10. Something related to early childhood education

Some specific domains (See the spreadsheet for more details):

  • concurrency in java/c++
  • socket and raw market data processing in c++
  • risk mgmt + derivative valuation?