I started a new job about 4 months ago. One of my soft criteria was a company not using Python. I love Python, but it's been my primary language since 2001, and I'd been feeling like I wanted to stretch. And I had this hypothesis that the jumping in the deep end was the best way way to do that. So what did I learn?
Just looking at the source you call might not be enough: sometimes you have to follow the thread further. The faraday-http-cache middleware looked like it'd work out of the box, so long as my service supplied an ETag.
Today I discovered that it was hashing the entire set of request headers to determine the cache key, not just those that might Vary.
There's also this magical thing that Ruby seems to do, where you can omit the {}
around your keyword arguments, and it collects them up into a single hash. I'm not sure how I feel about that.
Things I've developed some expertise with in the last four months: Redis, DynamoDB, Node.js, RabbitMQ, Heroku. Some of these were all new to me (DynamoDB, Node.js, Heroku), and some of them were tools I'd used, but only as a consumer (ie, RabbitMQ by way of consuming Celery).
Something that's bitten me a few times -- although I'm slowly internalizing it -- is the fact that while *Person
is a pointer to a Person
struct, *Individual
is not a pointer to "any type implementing Individual".
*Individual
is a pointer to the interface itself.
A parameter that's an interface type doesn't tell you anything about whether it's a pointer or a copy. Either can implement the interface, depending on how the methods are declared.
There are lots of things that seem obviously wrong only once you discover the problem. Up until that point, you can read the code over and over as you work on it, and never see the issue.
It seems like Javascript Promises are particularly prone to this sort of sharp edge.
Today's example:
this.worker.doSomething()
.then(...)
.catch(...)
This works fine, so long as this.worker
actually has a method that returns a Promise called doSomething
.
In our case, this
used to have a worker
, which became worker()
, which meant doSomething()
stopped working.
I like the way Promises make my code look at a glance, but they fundamentally provide illusions: they make async code look synchronous (linear), and they use words like catch
that we're used to seeing with different semantics.
It'll be interesting to see how I feel about them in another 90 days.
DynamoDB is a hosted NoSQL database, part of Amazon AWS. We're using it as the primary data store for our messaging system.
Dynamo pricing is based on three factors: dataset size, provisioned read capacity, and provisioned write capacity. Write capacity is by far the most expensive (approx 3x as expensive as reads).
The provisioned I/O capacity is interesting: as you reach your provisioned capacity, calls to Dynamo being returning HTTP 400
responses, with a Throttling
status. This is the cue for your application to back off and retry. I'll come back to throttling shortly.
When you start out -- say with no data, and 3000 reads and 1000 writes per second -- all of your data is in a single partition. As you add data or scale up your provisioned capacity, Amazon transparently splits your data into additional partitions to keep up. This is one of the selling points: that you don't have to worry about sharding or rebalancing.
It's not just your data that gets split when you hit the threshold of a partition: it's the provisioned capacity, as well. So if you have your table provisioned for 1000 writes per second and 3000 reads per second, and your data approaches the capacity of a single partition, it will be split into two partitions. Each partition will be allocated 500 writes per second and 1500 reads per second.
DynamoDB works best with evenly distributed keys and access, so that shouldn't be a problem. But it could be: if you try to make 600 writes per second to data that all happens to live in a single partition, you'll be throttled even though you think you have excess capacity.
Provisioning that I/O capacity is important to get right: it's not sufficient to turn the dial all the way to 11 from day 1. That's because Dynamo will also split a partition based on provisioned I/O capacity. A single a partition is targeted roughly at the 1000/3000 level, so doubling that to 2000/6000 will also cause a split, regardless of how much data you have.
Splits due to provisioned I/O capacity -- particularly when you dramatically increase the capacity for a high volume ingest -- are the source of dilution.
"Dilution" is the euphemism Amazon uses to refer to a situation where the provisioned I/O is distributed across so many partitions that you're effective throughput is "diluted". So why would this happen? Well, remember that a partition can be split when either data size or provisioned I/O increases.
So if you decide that you want an initial ingest to complete at a much faster rate than your application is going to sustain in production and increase the provisioned I/O to match, you're effectively diluting your future I/O performance by artificially increasing the number of partitions.
Whomp whomp.
RabbitMQ topic exchanges are a powerful tool: they let you publish messages to an exchange and route it to zero or more queues in parallel. They're rapidly becoming the Rabbit tool we turn to first; we've used them as fanout exchanges by "accident", and been happy for the flexibility later.
This weekend we dropped some data on the ground, though, because they route to zero or more queues. If you publish to a routing key that doesn't have a queue bound, the message is dutifully routed... nowhere.
You can set an Alternate Exchange for any Rabbit exchange for handling just this situation: any message that an exchange can't handle will be routed to its alternate exchange, if set.
I know now that when using an exchange, you should always configure an AE for it. It gives you visibility into whether you have routing bugs, as well as a way to recover from those bugs (by replaying the messages into the correct queue).
This week I had to add caching to a specific HTTP endpoint. The endpoint is implemented in Express/Node.js, and unfortunately has caching semantics that mean a general purpose cache (i.e., Varnish) isn't appropriate. If I were still working in Django, I'd have either decorated the function, or written a piece of middleware to handle the caching. So my first exploration was in that direction.
I read the Express docs, and then I read the Express code. I'm not in love with NPM's dependency resolution model, but I do like that all of your dependencies are just in a sub-directory for easy perusal: no site-packages
, dist-packages
, virtualenv, etc to explain.
I learned that to do the sort of wrapping that's so natural in Django with Express middleware meant overwriting a bunch of methods on response
to catch the output as it streams by. If you just override end
, you're probably too late.
For an example, check out the compress middleware. It makes for some difficult reading.
I'm more aware today that there are layers to learning: there are facts, and there are implications. I knew the fact that Express was good for streaming responses, but hadn't considered what the implications of that were on how I'd write collaborating code.
When the first European Americans climbed Grand Teton they discovered a rock shelter has been built near the summit by an earlier native climber.
13 degrees centigrade is known as "banana" as that is the temperature at which they are kept when they are being shipped.
If you are close enough the raven and the crow can be distinguished by the fact that a raven has talons and the crow does not.
Hollow as an American place name means a valley surrounded by mountains and was first employed by fur trappers.
Dessert a la mode apparently means with ice cream. But more investigation is required.
The Moon is unique in the solar system in terms of its size ratio to its planet. It is the largest moon by ratio (although not in absolute size, there are larger moons around the gas giants).
"These twice five figures" with which we count are clearly related to our five fingers[0].
But why five? TIL that it's a few hundred-million-year backward compatibility issue:
Because there are only five distinct Hox-encoded domains across the limb bud there is a developmental constraint prohibiting the evolution of more than five different types of digits.
[0] at least when counting moving objects, such as cattle; bakers, whose objects tend to stay where one left them, use nicer rectangles, and hence prefer dozens to tens.
The American Bison is its own species (Bison bison) and is not related at all to buffalo. Essentially all references to buffalo in North America are incorrect.
When shaving you should shave in the direction that the hair is growing (where was my father with this helpful advice?).
You can sharpen serrated knives, you can use a rod on the bevelled serrations or buy a specialised sharpener (I went with the latter).
Resolving two clauses with more than one pair of complementary literals always yields tautology.
(formally, the absorption works almost identically to the well-known physical notion that ∂∂=0 "boundaries are unbounded". I have not yet learned whether there is anything more to this correspondence than the simple observation that a logical formula represents –cf Karnaugh– a "Truth polygon"...)
Aristotle's Organon roughly translates to "instrument, tool".
Was "Notation as a Tool of Thought" a conscious tip of the hat to the Stagirite?
(much less likely, but possible: was this title simply a metasyntactic variable Aristotle's students used to refer to his lecture notes? — cf fr. truc)
Cats are all long-sighted due to their large eyes, so during the day they rely on their sense of smell and whiskers to find things near to them.
Like most cats they have black and white vision. Their vision is much more effective at night.
IP addresses can be sorted with sort -V
. (It's the option for version numbers.)
bis quinque figuris: (Carmen de Algorismo)
Baked beans are cooked after being sealed in the can using a pressure cooker.
I'd always thought Landin's "Program Machine Symmetric Automata Theory" (Machine Intelligence 5) rather imaginative:
"Suppose a directed graph with coloured edges is drawn on a plane, and another one drawn on the surface of a sphere. Imagine the sphere rolling ... on the plane, but constrained in such a way that the point of contact follows ... both graphs ... synchronizing the vertices and matching the colours.
TIL that mid-XX fire control actually involved computations via rolling (albeit continuously) spheres on planes.
(to paraphrase Körner: that these have been obsoleted by digital systems probably says more about human technological than moral progress over the last several decades)
Markdown uses significant whitespace at the end of lines to denote
tags.
The main tenet of Sex-postive is all consensual sexual activities are fundamentally healthy and pleasurable.
Had heard the phrase (and I guess understood by context) but nice to have some manifesto around it.
Ernie Hudson's character Winston in the original Ghostbusters was originally in the whole film before being written out before shooting started.
In the original Civilization, there was a signed integer bug that made Gandhi overly aggressive. Normally Gandhi's aggression level is a very low 1
. If you are playing a democracy, your opponents' aggressions are reduced by 2
.
But what should have been Gandhi's aggression level of -1
in that case was treated as 255
, making him the most aggressive possible.
/via http://kotaku.com/why-gandhi-is-such-an-asshole-in-civilization-1653818245
Yes, I'm slow to the party. It was just today that I learned that TIL means today, I learned
IP addresses can be sorted with sort -V
. (It's the option for version numbers.)
One of the largest and wealthiest slave owners in Tennessee was Sherrod Bryant, an African-American.
Abraham Lincoln is in the National Wrestling Hall of Fame for a record of 299-1.
This shocked me as I have always thought of Abraham Lincoln as a feeble and sickly man, which was true of his later years as President and his paintings/photographs reveal a thin and weak man. I never considered how he would have been as a younger man.
The point on the Earth's surface furthest from the center of the Earth is NOT Mount Everest.
Due to the equatorial bulge of the Earth, caused by its rotation, Chimborazo in Ecuador is the furthest.
Tom Anderson (of MySpace fame) was a fellow hacker and friend of Bill Landreth.
Elvis was a huge Monty Python fan.
The chance that an arbitrary permutation doesn't have any fixed points tends to 1/e.
(is this any relation to the 1/e that shows up in optimal stopping heuristics?)
MRIs work by aligning the spin of the hydrogen atoms in the water in your body with a magnetic field.
Hold down Option when clicking on sound menu bar item. It allows you to change the input and output devices from the menu bar.
In the XIX, FMACs were done with springs (and clocked via handcrank):
I saw professional singers using the trick of using your skull as a resonator for a tuning fork (something I did previously know you could do) to hit a particular note during a piece where the voices were accompanying an ensemble with pure tonal vocalisation.
the null program, at least by certain measures, has manifold ramifications — more than the typical filesystem.
You can look at the history of a single line in git by using:
git log -L<line number>,+<no. of lines following (or before if negative):<path to file>
Red paint is the cheapest due to the abundance of iron oxide, which is in turn is due to the balance of strong and weak atomic forces that lead to iron oxide's stability.
Iron oxides are not only abundant, they globally occur on a geologic scale in large, layered "Banded Iron Formations"; what I was taught about these seems to still be the conventional wisdom, namely that they're a highly visible side effect left over from when the biosphere discovered photosynthesis.
The West Africa country of Liberia was founded as a colony of America in 1822 by American Colonization Society.
It's capital, Monrovia, was named after the then US President, James Monroe.
Some online text editors prefer
tags over paragraphs (Blogger I'm looking at you but not just you) because they allow you to run inline elements across multiple "paragraphs" without having to worry about the HTML rules that terminate an inline tag if it crosses a block tag boundary.
Growing up, I assumed the Tropic of Cancer and Tropic of Capricorn were defined at a set latitude, but I recently discovered they are, in fact, astronomical and vary over time.
And I just learned that the Arctic and Antarctic Circles are similarly astronomical and vary over time.
Once upon a time, an investor related an old story about working on some government contract involving vector displays costing thousands, if you could source them at all ... and then being shocked to discover Vectrexen being sold in Sears for under USD 200. TIL that in the early 80's, game companies didn't just buy displays, they did some serious engineering to meet their price points via hw/sw codesign.
From when "spaghetti code" was meant literally: how adders could multiply (w/out logs):
Boyell, "Programmed Multiplication on the IBM 407" (1957)
To prevent Homebrew packages from being accidentally upgraded use brew pin $FORMULA
and brew unpin $FORMULA
respectively. This fixes a long standing issue that I've had with Postgres databases being accidentally upgraded.
The J language hashtag has a high (proper!) ratio of models to lines of source code.
(upon investigation, #JLang might have a maximal intersection of "fashion models" with "source code", not just for programming hashtags, but also —despite Amazon's best efforts at searchspam— for the web in general)
You can do a discrete Fourier transform in just one short line of J:
dft=: +/ .* ^@o.@(0j2&% * (*/ -)@i.)@#
Courtesy of this post by Andrey Paramonov which I indirectly found via a link from Dave's recently TIL post on the jlang
hashtag.
The Celsius temperate scale was originally reversed, with 0º at the boiling point of water and 100º at the freezing point.
It was flipped to the direction we know now after a couple of years, after Celsius had died (although numerous people seem to have independently developed it).
According to Hamming, the naming instance of Gibbs' phenomena was due to investigating an unexpected behavior of Michelson's mechanical fourier analyzer ... not just chalking it up to experimental error.
("The Art of Doing Science and Engineering", pp109-110)
Cayley not only introduced the term "tree" (in the CS sense) to the english language in 1857, he did so as a computer scientist:
Not quite today but the Highlands of Scotland are the other side of a rift line from the Lowlands.
Also England and Wales were already a landmass (Avalonia) before they collided with the island chain that is now Scotland.
Intensively reared pigs are slaughtered 17 weeks after their birth
xz is a compression format that is based on the old 7-Zip archive format but with added Unix support.
Also poor old Bzip2 is being retired and GNU tar
will autodetect the compression used on a file, no more z or j!
To navigate between tabs in Terminator on Ubuntu use Ctrl and Page Up and Page Down.
Healing wounds, and in particular broken bones, requires higher than normal calorie consumption.
If you want a textarea element to be selectable but not editable (e.g. for cut and paste), use the attribute readonly
and don't try and disable it.
When Amazon Music notifications get stuck in your Android notifications stream just block the app from notification settings and then unblock.
The insert/overwrite label in the Libre Office status bar is blank on OSX
The Welsh word 'Glas' can mean blue, green or gray. Which makes it sound like really it is meant to describe the colour of the sea.
However it turns out that several languages use the same word for blue and green and that blue is one of the last colours to be given a specific designation.
Stonehenge's "Bluestones" are gray and Glasgow means "Green Hollow".
If Windows Explorer is dead you can restart it via Task Manager's Run menu option (off of File).
Angular directives must have closing tags otherwise only the first directive will appear.
Mocha coffee is named after the Yemeni Red Sea port of al-Mukha where the coffee bean bales were traded and shipped.
The Shard's nickname was coined by English Heritage in complaint about the design (a shard of glass through the heart of historic London) and thereafter adopted by the building's marketing department.
Celtic crosses feature a ring around the centre of the cross to support the carving of larger crosses. Without the ring, the arms of the cross fall off after a few years.
Goldman Sachs has an organisation on Github with one project. A Collections library that doesn't accept external contributions.
A person can run into the centre of a small tornado and keep pace with the calm centre of the vortex.
The tornado has to be quite large and powerful to knock you off your feet and generally the danger is the dirt and stones that the tornado has in its column.
Crocodiles eat stones to act as ballast and to help digest food.
2010 was apparently the year for English wines
The "Alabama Song" was originally written (in english) for a Brecht work (which was then performed in german) in the 20's, before being covered by the Doors (in english) in the 60's.
In moments of ambiguous requirement desperation, my team and I have sarcastically thrown around the term defect-driven development (shorthand: DDD). TIL "DDD" is a previously established acronym for Domain-Driven Design which is an actual (complex) approach to software development. #ddDesign
The sequence:
0 → algebra scenes → hollywood movies → profit → 0
fails to be exact at small values of "profitable":
It's My Turn (1980) Snake Lemma
The first log tables were laboriously hand calculated once, and then apparently cut'n'pasted from one application to another for the following three centuries:
all logarithm tables for three hundred years were borrowed from Mr. Briggs’ tables by reducing the number of decimal places
Today, computation is too cheap to meter and one can program the necessary calculation, even without import math
, in a matter of minutes. Obtaining the patterns required to make the insights necessary to get to \( \mathbb{C} \) when given \( \mathbb{N} \) ... might take a bit longer.
# after http://www.feynmanlectures.caltech.edu/I_22.html#footnote_source_1
_=min
sqrt = lambda n: _( cvg(nstep(n1))(n1)
for n1 in [float(n)]
for cvg in [lambda f: lambda x:
_( x if y==x else cvg(f)(y)
for y in [f(x)] )]
for nstep in [lambda n: lambda g: (n/g+g)/2] )
if __name__ == "__main__":
def D(x):print(x);return x
# regenerate table 22-1 from feynman LoP
n = 10
for i in range(16): n = sqrt(D(n))
# check precision after inverting
for i in range(16): n = n*n
print(n,n - 10.)
On Hills and Dales, although published almost a century and a half ago in Philosophical Magazine is surprisingly (at least in my search engine bubble?) difficult to find in full text. Best I could do was p.233, paper XLIII in a collection of Maxwell's works.
Is there an anglophone equivalent of Gallica?
Edit: Cayley's 1859 On Contour and Slope Lines was, happily, more readily available.
"if a practitioner [of Brazilian Jiu-Jitsu] receives his or her black belt at 19 years old, the earliest they could expect to receive a ninth degree red belt would be at the age of 67."
TIL that not only is a « zapaska » a "stepney" in english
(thank you, Dyadya Google) but also that a "stepney" is neither cisatlantic nor translatlantic, but transpacific. (thank you Auntie Beeb).
By way of Bunnie, TIL Gordon Moore may not have predicted current fashions (and was a little pessimistic about the size of smartphones), but he definitely knew —a half century ago— how pursetop computers would be marketed:
The Tongan Scrubfowl incubates its eggs by burying them in volcanic ash that is warmed by the volcano.
TIL @michaelbolton is a programmer. I failed to learn, however, if he knows what "PC LOAD LETTER" means.
TIL how to trick a neural network into thinking a panda is a vulture
Blended whiskys were more desirable than single malts prior to the 1960s due to the consistency of their flavour.
A bath chap is the roast lower jaw of a pig including part of it's tongue.
Many things about the American electoral system but the few standouts are:
Transport planners think about crush loading. Also, experienced crush load due to poor planning of my cross-London journeys.
Brazil has thirteen operational political parties (and more in absolute terms) and they are mostly aligned to patronage and regional organisations.
Wild ginseng is an endangered species due to poaching and intensive foraging with quality wild ginseng reaching prices of $850 a dry pound (U.S. presumably)
Angela Merkel speaks fluent Russian and Vladamir Putin speaks fluent German.
TIL about a Cousot et Cousot paper on Galois Connections ... well, actually it's (as one might guess from the topic?) a pair of papers: one of which is the expanded concrete representation, the other of which is a compressed (albeit still containing most of the important bits) abstract representation.
The soviet-era Elbrus computer line was based on the same B5000 that many software people consider "an improvement upon nearly all its successors" (unfortunately for Burroughs' reception in the marketplace, their peripherals may not have been so reliable: worse is better, except when it isn't).
TIL the Elbrus tradename is still in commercial use, although these days it looks as if it's being slapped on systems running Linux on SPARC derivatives instead.
However, if the RF equivalent of "orange book" systems buyers still can source something with the safety properties of the old MCP lineage, it might help explain the recently infamous « русский хакер ». After all, if you live in a wooden house, with roughly capability-based security, while your neighbor's glass house is chock full of legacy x86 and C/C++, it must take an awful lot of willpower to avoid throwing stones...
Celery is an allergen
Obsession and Possession originated as terms to describe phases in siege warfare.
In a world where Poe's Law appears to reign supreme, one can always peruse abstract comics. They have the graphical syntax of comics, but lack semantics. Might they be the grandchild of Dada?
Milo Yiannopoulos is British and originally lived in Kent.
London Fashion Week uses a fleet of Jaguar XF cars
The moon has a magnetic field but no poles, the majority of the field is generated by impact craters on the crust.
POST TENEBRAS LVX
TIL the Republic and Canton of Geneva has a nifty motto, thanks to following the architectural references in Geneva's entry in the grand tradition of #2 videos:
"Geneva second | Genève interpelle Donald Trump"
c'est toujours le plus intelligent qui cède
TIL about the Turpan hydraulic engineering. I found it especially interesting, because just behind my house is one of our local (above-ground) canals, conducting glacial melt from the alpine valleys above to the grapes and apricots growing below.
Online and offline come from nautical terms.
Mexican Coke refers to Coke that is made with cane sugar rather than corn syrup.
(not from today, but...) it's not just Mexico and the US; "coca-cola" is more a brand than a specific flavor.
[Edit: ... and for people who live in sufficiently cosmopolitan parts of the US, sugar-sweetened coca-cola can be found]
California has a grizzly bear on it's state flag but the species is extinct in the state.
Gnome Keyring doesn't support Ed25519
SSH keys
Youtube: not only good for pseudorandomly encountering pop music from other cultures, but also serves as an unusually diverse mirroring medium for linux distros:
Older Siemens ES64U locomotives had sidelobes in the audio range during initial acceleration ... but someone decided this bug could be a feature, and so they played a distinct do-re-mi.
TIL Ernst Schröder would've have to wait nearly 80 years* —with the intervening invention of the computer — for his late XIX logical notation: (here intended as part of a universal formal "pasigraphy", as communicated to an international mathematical conference in Zürich) to find yet another application: as a simpler, if unfortunately far too earlier, alternative to predicate transformer semantics.
Giuseppe Peano was evidently a bit jealous of Leibniz' XVII ability to publish things like:
(to express commutativity and idempotency), for even in the late XIX he expresses himself anachronistically:
(Peano's versions of his symbols make better ping-pong paddles than our modern variants!)
Evidently Peano went even further than Schröder's Pasigraphy , proposing not only the formal notation for which we remember him, but even a creolized Latin to serve as a scientific conlang for informal argument ... but he wasn't foolish enough to attempt to revive greek!
One of the most surprising things about electronic mail is the ease with which misinterpretations arise
was a 1985 sentiment from RAND report R3283, "Toward an Ethics and Etiquette for Electronic Mail". Now I'm imagining ca. 2005 Odeo looking at email and exclaiming "hold my beer!"