numpy,scipy,pandas #cheatsheet

  • numpy provides efficient and large matrices
    • also linear algebra
  • scipy extends numpy
  • scipy compete against matlab and provides
    • mathematical optimizations including optimize.curve_fit()
    • stats
    • numerical integration
    • spline smoothing
  • pandas extends numpy
  • pandas provides key data structures like
    • time series
    • dataFrame i.e. spreadsheet
Advertisements

python usage in FI quant lib #Pimco

In one of world’s biggest fixed income buy-side firms’ quant library, the codebase is 3/4 c++ and ¼ python including pandas, numpy, machine learning, grid computing modules. I think this is similar to Macquarie FICC quant lib.

C++ is much faster, but data structures are very limited including STL containers.

I think the funds hold mostly bonds and mortgages. How about futures, IRS? Perhaps for hedging?

growth factor ] string/vector/hashtable #xLang

  1. std::string
  2. vector
  3. python list
  4. ArrayList
  5. hashtables

… all have algorithms to decide exactly how many percent more capacity to acquire during re-allocation. Usually it grows up to 2.0 in capacity:

python usage + risk system ] GIC #ChaoJen

Government Investment Corporation is a large buy-side. IT heads are keen about python as a first-class language, used for data analysis, in post-trade risk systems etc. Pandas is the most common module.

Python salary is not always lower than java/c++. My friend Chao Jen said python salary could be high if the team is demanding and deep-pocketed.

Is the GIC risk system mostly written in python? Chao Jen doesn’t know for sure, but he feels c++ may not be widely used in the GIC risk system. Might be a vendor product.

Hackerrank python tests can be hard to get 100% right, but Chao Jen said many get close to 100%.

[14] python (realistic) IV #misc

CV claims — “used py for sys automation”

Q: optimize – how did you optimize perf?
A: I didn’t need to for my scripts.

Q: try/catch + ….?
Q: immutable – what data types are mutable/immutable?
Q: threading – global interpreter lock?
A: never really needed threading
Q: xml – what xml parser did you use? ess477
Q: debugger? A: I didn’t need it

Q: read a config file?

Q: logging?
Q: DB access?
Q: command line arguments stored where? A: sys.argv?
Q78: exit normally? A: sys.exit()
Q78b: normally, “raise Exception” would print to stderr stream, but how do you direct that to a log file instead?
A: set sys.stderr stream. dive205 i.e. [[dive into python]]

Q: how do you handle newline diff between OS? ess114
Q: truth table? e68 (i.e. [[essentialRef]])
Q: how do you edit on windows and deply to linux?
A: samba, ftp. See also the post in pearl
— sys admin
Q: how do you pass input files to your py script? cook539
A: fileinput module is convenient.

Q: how is PYTHONPATH used with sys.path?

Q: another py — how do you execute another py script? Ess89 (i.e. [[essentialRef]])
Q: what command line options do you use often?

Q5: how do you launch an external program?
Q5b: how do you capture its output? c546 [[cookbook]] has a slightly advanced solution
Q5c: how do you call a unix shell command?
A: shutil module meets some of your needs

Q: exit with an error msg? cook540
A: raise SystemExit (‘msg’)

— data structures
Q: diff between listcomp and genexp? How would you choose?
Q: convert between list and tuple? cvc
Q: convert a list of objects to string and concat? see post in recrec
Q: split a string on multiple delimiters? cook37 explains re.split()
Q: iterating in reverse? cook119
==coding IV
Q: how do you print to a file rather than stdout?
A: e115 — print >> anOpenFile, ‘whatever’ # using the print KEYWORD
A: c144 — print (‘whatever’, file=anOpenFile) # using the print() function
Q: concat 2 lists?
Q: initialize a dict
Q: initialize a 2D array? m52 (i.e. [[perl to python migration]])
Q: walk a directory
Q: use a dict as a “seen”
Q: iterate a dict? mig87
Q: iterate a file
Q: interpolate into a string? ess115. c61 i.e. [[cookbook]]
Q: date/time usage (datetime module is OO;  time module is procedural?)
Q: trim both ends of a string? strip()

##minimum python know-how for cod`IV

Hi XR,

My friend Ashish Singh (in cc) said “For any coding tests with a free choice of language, I would always choose python”. I agree that perl and python are equally convenient, but python is the easiest languages to learn, perhaps even easier than javascript and php in my opinion. If you don’t already have a general-purpose scripting language as a handy tool, then consider python as a duct tape and Swiss army knife

(Actually linux automation still requires shell scripting. Perl and python are both useful additions.)

You can easily install py on windows. Linux has it pre-installed. You can then write a script in any text editor and test-run, without compilation. On windows the bundled IDLE tool is optional but even easier. For the ECT cycle – see https://stackoverflow.com/questions/6513967/running-python-script-from-idle-on-windows-7-64-bit

I actually find some inconveniences — IDLE uses Alt-P to get previous command. Also copy-paste doesn’t work at all. On Windows The basic python command-line shell is better than IDLE!

For coding tests, a beginner would need to learn

  • String common operations — learn 30 and master 20 of them
    • “in” operator on string, list, dict — is one of the operations to master
    • slicing on string and list — is one of the operations to master
    • converting between string, list, dict etc — is one of the operations to master
    • Regex not needed since many developers aren’t familiar with it
  • list and dict data structures and common operations — learn 30 and master 20 of them
    • A “Set” rarely needed. I never create tuple but some built-in functions return tuples.
  • Define simple functions
    • Recursion is frequently coding-tested.
    • multiple return values are possible but not required
  • “global” keyword used inside functions
  • if/elif/else; while loop with beak and next. Switch statement doesn’t exit.
  • for-each loop is useful in coding test, esp. iterating list, dict, string, range(), file content
  • range() and xrange() function – frequently needed in coding test
  • check 2 object have same address
  • null pointer i.e. None
  • what counts as true / false
  • · No need to handle exceptions
  • · No need to create classes
    • I think “struct-type” classes with nothing but data fields are useful in coding tests, but not yet needed in my experience.
  • · No need to learn OO features
  • · No need to use list comprehension and generator expressions, though very useful features of python
  • · No need to use lambda, map()/reduce()/filter()/zip(), though essential for functional programming
  • · No need to use import os and sys modules or open files, which are essential for everyday automation scripts

3groupsOf3digits #YiHai

Q: Find three 3-digit numbers with a ratio of 1:2:3;
These numbers pick their 3 digits from a range of 1 to 9;
All digits that form these numbers must be completely unique, i.e. you must use up all distinct 9 digits. For example, a set of 123:246:369 doesn’t qualify.

def failed_to_add(myset, digit): # return False when failed
    return digit == 0 or len(myset) == ( myset.add(digit) or len(myset))

def other_number_failed(myset, num): # returns False by default if everything good
    for digit in(num/100, num/10%10, num%10):
        if failed_to_add(myset, digit):     return True

def check(number1, myset):
    number2 = 2*number1
    if other_number_failed(myset, number2): return 
    number3 = 3*number1
    if other_number_failed(myset, number3): return 
    print number1, number2, number3, myset
    winners.append([number1, number2, number3])

winners=list()
nine_digits = tuple(range(1,10))
for digit1 in (1,2,3):
    last8 = list(nine_digits)
    last8.remove(digit1)
    for digit2 in last8:
        last7 = list(last8)
        last7.remove(digit2)
        for digit3 in last7:
            number1 = digit1*100 + digit2*10 + digit3
            if number1 > 329: break
            check(number1, {digit1, digit2, digit3})
print "-- lucky winners --", winners

my quicksort in python/c++

import random
# partition the given section by fixing the anchor
def partitionUsingFarRight(A, le, ri):
    pivotValue = A[ri] # immutable value
    pivotPos = i = ri
    while True:
        if (i <= le): return pivotPos
        i -= 1
        if A[i] > pivotValue:  
          swap(A, pivotPos-1, i)
          swap(A, pivotPos-1, pivotPos) 
          pivotPos -= 1
          
def partitionUsingFarLeft(A, le, ri): 
    # optional: swap a random object to the far left
    swap(A, random.randint(le, ri), le)
    benchmark=A[le]
    ret = i = le
    while True:
        if i == ri: return ret
        i +=1 #move pointer
        if A[i] < benchmark: # 3-step adjustment
            swap(A, ret+1, i)
            swap(A, ret+1, ret)
            ret +=1
    
def partition(A, le, ri):
    return partitionUsingFarLeft(A, le,ri)
def recurse(A, le, ri): 
    if le>=ri: return
    print 'entering partition()...',
    print(A[le:ri+1]), ' pivotVal =', A[ri]
    anchor = partition(A, le, ri)
    print '...after partition()   ',
    print(A[le:ri+1])
    recurse(A, le, anchor-1)
    recurse(A, anchor+1, ri)
def swap(A, x,y):
    tmp = A[x]
    A[x] = A[y]
    A[y] = tmp
def qsort(A):
    recurse(A, 0, len(A)-1)
    print(A)
 
qsort([222,77,6,55,3,11,5,2,88,9,66,22,8,44,1,33,99,7,66])

Above is py, below is c++


#include <iostream>
#include <vector>

std::vector<int> A{77, 11, 66,22,33,99,44,88, 77, 55, 0};
int const size = A.size();

void dump(int l=0, int r=size-1) {
	for (int i = l; i <= r; ++i)
		std::cout << A[i] << ' ';
	std::cout <<std::endl;
}

template <typename T>
void swap(int pos1, int pos2) {
	if (A[pos1] == A[pos2]) return;
	T tmp = A[pos1];
	A[pos1] = A[pos2];
	A[pos2] = tmp;
}

/*partition the region [l,r] such that all elements smaller than
pivotValue are on the left of pivotPosition
*/
template <typename T>
int partitionUsing1st(int l, int r) {
	T const pivotVal = A[l];
	int pivotPos = l;
	for (int i = l+ 1; i <= r; ++i) { 
		if (A[i] >= pivotVal) continue;
		swap<int>(pivotPos + 1, i);
		swap<int>(pivotPos + 1, pivotPos);
		++pivotPos;
	}
	return pivotPos;
}
template <typename T>
int partitionUsingLast(int l, int r) {
	T const pivotVal = A[r];
	int pivotPos = r;
	for (int i = r - 1; i >= l; --i) {
		if (A[i] <= pivotVal) continue;
		swap<int>(pivotPos - 1, i);
		swap<int>(pivotPos - 1, pivotPos);
		--pivotPos;
	}
	return pivotPos;
}
/*based on [[Algorithms unlocked]], should but doesn't minimize swaps!
Lime zone -- items smaller than pivot value
Red zone -- items bigger than pivot value
Ugly zone -- items yet to be checked
*/
template <typename T>
int partitionMinimalSwap(int le, int ri) {
	T const pivotVal = A[ri];
	// two boundaries exist between zones
	int redStart = le;
	// we start with redStart == uglyStart == l, which means item at le is Unknown
	for (int uglyStart = le; uglyStart < ri; ++uglyStart) {
		if (A[uglyStart] < pivotVal) {
			swap<int>(uglyStart, redStart);
			redStart++;
		}
	}
	swap<int>(ri, redStart);
	return redStart;
}

template <typename T>
void recurse(int l, int r) {
	if (l >= r) return; // recursion exit condition
	int const anchor = partitionMinimalSwap<T>(l, r);
	recurse<T>(l, anchor-1);
	recurse<T>(anchor+1, r);
}

int main() {
	recurse<int>(0, size-1);
	dump();
	return 0;
}

python: routine^complex tasks

XR,

Further to our discussion, I used perl for many years. 95% of my perl tasks are routine tasks. With py, I would say “majority” of my tasks are routine tasks i.e. solutions are easy to find on-line.

  • routine tasks include automated testing, shell-script replacement, text file processing, query XML, query various data stores, query via http post/get, small-scale code generation, simple tcp client/server.
  • For “Complex tasks” , at least some part of it is tricky and not easily solved by Googling. Routine reflection / concurrency / c++Integration / importation … are documented widely, with sample code, but these techniques can be pushed to the limit.
    •  Even if we just use these techniques as documented, but we combine them in unusual ways, then Google search will not be enough.
    • Beware — decorators , meta-programming, polymorphism, on-the-fly code-generation, serialization, remote procedure call … all rely on reflection.

When you say py is not as easy as xxx and takes several years to learn, I think you referred to complex tasks.

I can see a few reasons why managers choose py over java for certain tasks. I heard there are a few jvm-based scripting languages (scala, groovy, clojure, jython …) but I guess python beats them on several fronts including more packages (i.e. wheels) and more mature, more complete and proven solutions, familiarity, reliability + wider user base.

One common argument to prefer any scripting language over any compiled language is faster development. True for routine tasks. For complex tasks, “your mileage may vary”. As I said, if the software system requirement is inherently complex, then implementation in any language will be complex. When the task is complex, I actually prefer more verbose source code — possibly more transparent and less “opaque”.

Quartz is one example of a big opaque system for a complex task. If you want, I can describe some of the complex tasks (in py) I have come across though I don’t have the level of insight that some colleagues have.

When you said the python debugger was less useful to you than java debugger, it’s a sign of java’s transparency. My “favorite” opaque parts of py are module import and reflection.

If python (or any language) has excellent performance/stability + good on-line resources [1] + reasonable library of components comparable to the mature languages like Java/c++, then I feel sooner or later it will catch on. I feel python doesn’t have exactly the performance.

[1] documentation is nice-to-have but not sufficient. Many programmers don’t have time to read documentation in-depth.

[[automate the boring stuff with python]]

This book teaches just enough python features for the purpose. All

non-essentials are left out.

–sub chapter on selenium

the easiest way to use selenium and drive a browser, IMO.

–Chapter on Excel

text file spreadsheet converter

merge/unmerge

setting font in a cell

–Chapter on PDF:

combine select pages from many files

—-

Chapter on CSV + json

Chapter on task scheduler

Chapter on keyboard/mouse control — powerful

c++string tasks: IV+GTD

(Let’s be imprecise here… Don’t sweat the small stuff.)

We should be able to perform all of these using c-string, std::string (limited adoption since c++98), the standard string in java , c#, perl, python, php. This is a master list. Tolerate multiple names on Each task.

See basic tasks on https://bintanvictor.wordpress.com/2018/01/26/22tasksarraystrdictq-algoiv/

–STL
use string iterator with STL algorithms

–the easy
toUpper

–advanced
convert to vector and apply vector tricks
convert to std::string and apply tricks
equalsIgnoreCase
compareIgnoreCase
lastIndexOf
count how many times a substr occurs
sort content
endsWith

python – some relatively innovative features

I’m relatively familiar with perl, java, c++, c# and php, though some of them I didn’t use for a long time.

IMO, these python features are kind of unique, though other unknown languages may offer them.

* decorators
* list comp
* more hooks to hook into object creation. More flexible and richer controls
* methods and fields are both class attributes. I think fundamentally they are treated similarly.

python shortcomings for large enterprise app

– performance, efficiency. Compare — C statements translate to brief machine codes.
-? scalability
– threading – GIL, no precise control
– industrial strength commercial support. Only c++/java/c# has big tech powerhouses. Linux is widely adopted by enterprises, but is an exception that proves the rule.
– ? commercial auxiliary software to support the ecosystem, including IDE, debuggers, jars, code generators, profilers, … I feel python has these.
-? most enterprise apps are now in java or c# as these are proven, in terms of performance, resource scalability, commercial support etc.

python vs perl #my take

(Collected from various forums + my own, but ranking is based on my experience and judgment)

  1. OO
    1. – Not sure about perl6, but not many app developers create perl classes. Many CPAN modules are OO though. Python users don’t create many classes either but more than in perl. I guess procedural  is simple and good enough.
    2. – not sure about perl6, I feel perl OO is an afterthought. Python was created with OO in mind.
    3. – even if you don’t create any class, out-of-the-box python relies (more deeply) on more OO features than CPAN modules. Object-orientation is more ingrained in Python’s ethos.
    4. OO is important for large-scale, multi-modular development project. Python is more battle tested in such a context, partly due to it’s OO design, but i think a more important reason is the python import machinery, which is probably more powerful and more widely used than perl’s
  2. – Python can cooperate better with other popular technologies like DotNet (IronPython) and Java (Jython). See also python project in boost.
  3. – perl’s text processing (and to a lesser extent unix integration) features are richer and more expressive. Using every key on the keyboard is like using all 20 fingers and toes. I immediately felt the difference when switching to java. In this aspect, pytyon is somewhere between those 2 extremes.
  4. – perl can be abused to create unreadable line-noise; python has a rather clean syntax
  5. – As part of unix integration, perl offers many practical one-liners competing effectively with grep/awk/sed. Perl one-liners integrate well with other unix commands
  6. – Perl was initially designed for unix, text and as an alternative for shell script programmers. It’s still very good for that purpose. For that purpose, I feel OO offers limited value-add.
  7. – for unix scripting, perl is more similar to unix shell scripting at least on the surface, but python isn’t hard to learn.
  8. – I feel perl was more entrenched and more established in certain domains such as unix system automation, production support, bio-informatics
  9. big data, data science and data analytics domains have already picked python as the default scripting language. Perl may be usable but not popular
  10. – some say Perl the base language is much larger(??) than Python’s base language so probably it takes longer(???) time to learn Perl than Python but in the end you have a more expressive language???
  11. – CPAN was once larger than python’s module collection. If CPAN’s collection is A and python’s module colleciton is B, then I personally feel (A-B) are mostly non-essential modules for specialized purposes.
  12. – Python has better support for Windows GUI apps than Perl.
  13. I feel the open source contributions are growing faster in python
  14. There are more books on python

fopen in various languages (file input/output

–C++
ofstream outfile(“out.txt”);
ifstream infile (“in.txt”); // class template

–C
FILE * pFile = fopen (“myfile.txt”,”w”);

–php follows C
<?php
$handle = fopen(“a.txt”, “r”);
?>

— python:
outfile = open(“a.txt”, “w”) # semicolon is usually omitted

–perl
open (OUTFILE, “>>append.txt”) or die …  ### No dollar sign. parentheses are optional but help readability

–c# offers many convenient solutions —
TextReader rd = new StreamReader(“in.txt”);
TextWriter tw = new StreamWrioter(“out.txt”);

Alternatively, File class offers variations of
static string ReadAllText(string path)
static void WriteAllText(string path, string contents) //creates or overwrites file

–java
I have written so many of them but paradoxically can’t recall which class we need to instantiate

python features (broad) actually used inside 2 Wall St ibanks

If a python project takes 1 man-year, then the number of python features used would take 2 man-weeks of learning, because python is an easy-learning language. In such a project you probably won’t need fancy features like threading, decorators… Most of the work would be mundane.

After the initial weeks of exciting learning curve, you may feel bored. (c# took me months/years). You may feel your peers are deepening their expertise in c++ or wpf …

The tech quiz would be mostly on syntax.

========= the features =========
? lambda, closure, apply()
Everyday operations on list/dict/tuple, str, file object
** for-loop, conversion, filter(), map(), aggregation, argument passing,
Filesystem, input/output
Basics of module import (no advanced features needed)
command line processing + env vars
create functions –composite argument passing
config files and environment config like PYTHON_PATH

———a bit advanced and less used———
creating classes? fewer than functions, and usually simple classes only
list comprehension, generator expressions
basic platform-specific issues. Mostly unix
inheritance, overriding
threading
spawn new processes
decorators
* Python calling C, not the other way round. In financial analytics, heavy lifting is probably done in C/C++ [1], so what’s doable in python? I guess the domain models (market models, instrument models, contract models …) are probably expressed in python as graph-aware python objects.

[1] I have a book on option pricing using things like Finite Element Method. It seems to use many c++ language features (template etc) but I’m not sure if java or python can meet the requirements.

OO scripting languageS

I see more php developers create classes than Perl developers. I guess python developers are even more likely to create classes. Every language claim to support OO, but the litmus test is how easily (and how frequently) developers create user-defined classes.

(In the same vein, C++ template is a another thing regular developers have not fully embraced. Java generic class is yet another.)

Unlikely Java/c#, these scripting languages won’t deliberately make your life difficult if you don’t “embrace” OO.

Compared to perl and php, I feel python OO is simple, clean and “full-service” (not in the call-girl sense:). Incidentally, built-in python “features” are more object-oriented than in perl and php.

However, I still feel many python applications tend to be more procedural than OO. When facing a choice, I guess many developers feel procedural is simpler even thought python OO is not complicate. If a simple tool does the job, then no motivation to change.

What about multi-module team development? Look at perl and C projects. Many team still stick to procedural.

python: very few named free functions

In python, Most operations are instance/static methods, and the *busiest* operations are operators.

In the “neighborhood”, Free-standing functions aren’t allowed in java/c# but the mainstay of C, Perl and PHP.

— perl-style free functions are a much smaller population in python, therefore important. See the 10-pager P135[[py ref]] —
del(…)
len(..)
map(), apply()
open() 
min() max()

— advanced free functions such as introspection
repr() str() — related to __repr__() and __str__()
type(), id(), dir()
isinstance() issubclass()
eval() execfile()
getattr() setattr() delattr() hasattr()
range(), xrange()
?yield? not a free-function, but a keyword!

xx python – first lesson

—-first lesson ( 4 – 16 hours)
variable scope, object lifespan
sequence data structures
basic slicing to query, modify a sequence
pbref, pbclone
object assignment
basic operations on/among tuple, list, dict

—-Lesson 2 — classes. (Fundamental language feature like c++ templates, but some teams don’t create custom classes at all!)

ctor
methods
creating class, using it
inheritance?

–LGpp
file access
if/while/for
regex

c#delegate:closest ancestors]other lang+py

Update — I now feel it’s more practical to separate the 2 (or more) “streams” of thoughts of labmda vs multicast events. The write-up below was written without that seperation. Unnecessarily complicated.


a unicast delegate type =~= a java interface type with a single method, without method body. [1] I no longer subscribe to this.
a unicast delegate type =~= a typedef for a function pointer. This typedef can apply to many functions.

a “delegate” is an instance of a particular delegate type. If we have to build on the java interface idea, then ….

a delegate instance =~= an instance of a stateless concrete single-method class implementing that interface in [1]

a delegate instance =~= a particular stateless functor. Specifically an instance of a functor-wrapper. Note An instances of a functor type is sometimes stateful (see Scott Meyers), but an instance of a delegate type is always stateless.
Update — As explained in http://msdn.microsoft.com/en-us/library/system.delegate.aspx ….
a delegate instance =~= instance of a functor-wrapper having a permanent pointer to the target object myDog, so it could later invoke myDog.jump()

a delegate instance =~= python method object bound to a host object? But stateless??

a delegate type is a distinct type (with a name), a subtype of Class Delegate. Delegate type names typically start with D like “DSomething”

The clue to see the link between a delegate type and a delegate instance is the instantiation expression new DSomething(meth1)

Java’s interface is not really an extension of the c++ functor. C# delegates build on top of java interfaces AND c++ functors.

python module #simplified learn`curve

One of the books I read (cookbook?) point out that module is a deceptively simple, but less-understood feature of python…

In python (just like in perl) the module feature is simple to use. As a beginner, you need to focus on usability, not those fancy stuff behind the scenes. For any module, you just need to know

1) how to Import, Use, and Read other people’s tricks importing/using a module.
1b) learn module hierarchy

2) learn how to create a module.

learn sys.path (the module search path)? maybe
PYTHONPATH? maybe
dir(anyModule)? no need
the __dict__ thing? no need
reloading? no need

dict – at the heart of python, perl and javascript OO

Q: why is dict so central to python OO?
%%A: method object — python methods are easily modeled as code objects (just like fields), therefore more dict-friendly. Recall the regular dict stores key and value objects.
%%A: ultra-virtual — both field and method lookup is a runtime dict search. Note java/c++ does runtime search only for methods, not fields.
%%A: namespace — is a kind of dict. Namespaces are central to inheritance (and presumably overriding)

dir(), vars(),__dict__  … often returns something like a dictionary.

Note the dict in this context is a kind of “kernel-dict”. On the other hand, the user-defined dict is probably a “userland dict”. They share the same syntax. Implementation wise, they may differ. kernel-dict is probably memory efficient and in implemented in C. These various types of dict probably implement a consistent “interface”.

When people say an “import sys” creates an object named “sys”, it probably means a dict created in memory.

———–
In Perl OO, a hash is at the heart of every class. This is the “userland dict”.

———–
In javascript, “associative array” is closely related to (if not the basis of) user-defined objects

every python thingy is an object with a type #del

I like this particular insight in [[py ref]] (i.e. “python essential reference”) though I can’t easily verify it in code — Every thingy in a python program is an object with a type. (There might possibly be some exceptions but this is a rather good starting point.)

* An object is defined, same way in C++, as a storage_location, with an (obviously) immutable address.
** content could be mutable.
** Even an int is an object, unlike java

* Every storage_location (Say ObjectA) has a specific data_type. Required by python interpreter, java compiler or c++ compiler… That specific data_type is technically a Type OBJECT.
* [[peref]] made it clear that Type tells us the features supported by this storage_location. Features include methods and fields…

The thing about the python language is that Everything seems to be an object with a type. Here are some special objects and their types
$ a free function. This OBJECT has a __name__ attribute. Try qq/dir(len)/
$ a bound instance method. Such an OBJECT has an im_self attribute.
$ a bound class method. Such an OBJECT has a __name__ attribute
$ (There’s no such thing as a bound static method — nothing to bind.)
$ a regular instance method. Such an OBJECT has a __self__ attribute pointing to the host OBJECT
$ a regular class method. Such an OBJECT has a __name__ attribute
$ a regular static method. Such an OBJECT has a __name__ attribute

Warning — these special objects each have a distinct data_type but not by type()! It’s worthwhile to step back and recognize these special OBJECTS as separate data_types with their own features. The “feature set” difference between any 2 data_types is reflected in various ways (such as the attributes…)

Footnotes
* the built-in type() function is an almost-perfect way to see the data_type of a storage_location, but I don’t think it reveals all the details I need in a foolproof way.
* The “type” OBJECT is comparable to type_info objects and java “class” objects.
* What’s the type of the “type” OBJECTS? We don’t need to know.

import+package+namespace ]py/c++/java/perl

Namespace is a common challenge, and a common feature. Physically, module files invariably live in a hierarchical file system.

Perl’s solution revolves around the concept of package and symbol table….

java’s solution is rooted in fully qualified type names. Every named type has a FQTN. They naturally form a hierarchical namespace tree.
–c#???
–Python
Python instantiates a namespace object “os” (or sys, re) when you say “import os, sys, re”, so you can use the dot notation like os.path. Thanks to this “object”, Python’s introspection, instrumentation and meta programming capabilities shine through.

Python’s “from os import environ” imports the environ variable into the current namespace, just like c++ “using std::out”

–C++ namespaces are well-covered in [[absolute c++]], and also concisely covered in effC++
using namespace std; // imports all the vars and funcs into “here” so we don’t need to say std::cout
using std::cout; // imports just one var

However, in both cases your own “cout” variable will clash with the cout imported.

There’s a 3rd usage of “using myBaseClass::method2” on P413 of ARM

Note, “using” can be nested in  a class.

i-socket operations in perl, python and c

(“i-socket” means internet socket. Our discussion doesn't cover UD-socket ie unix domain sockets.)

1st) and easy lesson – both perl and python provide wrappers over C-level syscalls, so function names are identical across C, perl and python.

2nd) Lesson – python uses a socket object as the Dictator (remember Benevolent Dictator For Life?) whereas Perl puts a socket FILEhandle as the first argument to those functions.

Like Perl, C puts a socket FILE descriptor as first argument to those functions. However, the C file descriptor is different from a Perl file handle —
– C file descriptor is an integer either for a socket or for a disk file
– Perl file handle is internally implemented like c++ i/o streams. As compared in the post on [[pipes, streams, io classes]], all of these are built on the PIPE concept

However, both C and Perl treat a socket like a file, to support read/write.

python features used in a web app (django

(Plone is too heavy-weight and over-kill, just like zend. Django is light-weight.)

MVC – model/view are classes. Controllers may/may not be classes.
user-defined class hierarchy
per-instance method-redefine
per-instance attribute-add/delete
lambda, apply — functional programming
try/catch – heavily used
decorators – Django-specific
introspection (basic meta-programming)