Interesting Links for 05-10-2020

[andrewducker]

The technology needed to make a game as beautiful as "Ori And The Will of the Wisps" work on the Switch makes my brain ache

(tags:technology video games computers animation )

The order of operations in mathematics does not quite work the way you expect it to

(tags:mathematics language )

How universities tricked students into returning to campus

(tags:university UK pandemic OhForFucksSake )

Trump-linked consultant tied to FB pages warning election will cause civil war

(tags:OhForFucksSake civilwar USA )

The Rules for Displacing an Ailing Presidential Candidate

(tags:law voting politics USA viaDanielDWilliam )

'It's like church:' The movie theater still hosting audience-less showings of 'The Rocky Horror Picture Show'

(tags:movies pandemic viaTheFerrett )

Testosterone Alone Does Not Cause Violence - Scientific American

(tags:testosterone behaviour dominance psychology violence )

It's official: They have come for the Communists

(tags:fascism USA OhForFucksSake immigration )

Home Secretary Priti Patel declares herself against both lawyers and those who do good.

(tags:UK immigration evil OhForFucksSake conservatives )

UK Covid-19 data underreported due to file size limit exceeded

(tags:incompetence pandemic computers )

Comments

[momentsmusicaux]

Not quite file size.

They were using EXCEL and it overflowed the maximum number of columns.

EXCEL.

[andrewducker]

More importantly: COLUMNS! Excel is fine for storing a few pieces of information in a long list. But why would you store one person per column, rather than per row (where you can have millions!)

[momentsmusicaux]

Also, it's only 2-dimensional! Surely that's not enough! How are you going to group results by different levels of region? Gah. The whole thing is just mental. How did people this fucking useless end up running this?

[andrewducker]

Oh yes, you'd want all sorts of useful tools to extract and visualise the data, I'd have thought. Although I don't know how much they actually have. Actual data for the track/trace data capture may be fairly minimal.

(You can do very swish data grouping and filtering with pivot tables too. But I don't know if that would cover everything you'd need)

[channelpenguin]

OMFG! How? Just how? I mean any software dev competent with data design and use would just never DO that (anyone pro, IMO, but you do meet some right chancers and cases....) Cronyism at its best (worst)?

[channelpenguin]

got a link for those details? I want to pass the OMFG WTF-ery joy onto other devs!

[momentsmusicaux]

It's all over twitter.

[channelpenguin]

ah,. dont Twitter. Found it though!

[momentsmusicaux]

Latest is that one (or more?) labs were sending their data as CSV files. The files were really long, and Excel ran out of rows.

*FACEPALM*

[andrewducker]

Excel can handle millions of rows. It's columns that are limited to 16,000.

[momentsmusicaux]

I just googled and I got told '1,048,576 rows by 16,384 columns'.

That's technically 'millions' in that it's more than 1 million... ;)

[andrewducker]

And now I'm reading that actually the sheet had every test ever done in it, which *was* over a million.

I shall look forward to the truth coming out!

[momentsmusicaux]

> And now I'm reading that actually the sheet had every test ever done in it, which *was* over a million.

WHY??? *wails*

[andrewducker]

Really simple systems bodged together in a hurry where nobody really understands how they work or what the limits are that will make them break?

[momentsmusicaux]

But anyone who's worked with data would tell you Excel isn't up to that job! What kind of numpties were doing this?

[andrewducker]

Cheap ones.

(I suspect the developers didn't expect every record to be uploaded each time, just that day's set. Which would be much smaller. And the users knew the back end would deduplicate, so didn't bother cutting the dataset down. And when you put those two things together...)

[momentsmusicaux]

Sending CSV files rather than building an API for labs to send the data is a quick bodge. That's fine.

This though...

[andrewducker]

Oh yes. They've built in an unnecessary dependency on Excel somewhere in the middle. I bet someone competent will now replace it with a three line script.

[jack]

I was thinking "I bet they don't..." :(

[channelpenguin]

OMFG2! Every database can handle those. It is a bog std ETF task to get the data into suitable shape for your needs. It is absolutely normal normal data-shovelling like probably literally tens of thousands of us have written ridiculously many times in our careers. Fuck. bet next stage is they try to roll their own CSV parser (AND FAIL BIGSTYLE - bad plan)

[channelpenguin]

.. unless they already failed that and that is why they went Excel?

[zz]

the people responsible for the students coming back should be flogged round pollock halls 3 times.

[mlknchz]

I may be a minority, and I may be branded as a fascist for saying this, but I have as much contempt for communists - and for the same reasons - as I do for fascists.

However, doctoring that quote is wrong in a LOT of ways.

[snippy]

I used to drive by that theater every week; I've been to movies there (I particularly remember seeing The Blair Witch there). So glad they're continuing their tradition.