The Daily Parker

Politics, Weather, Photography, and the Dog

And hey, a bug

Not five minutes after my last post, I discovered a completely borked feature, caused by a change to the way Azure.Data.Tables executes queries.

The Daily Temperatures feature stores data in the same table as the History feature. Each row represents a weather report, where the table partition key is the weather station identifier and the row key is the date and time of the report. So, for example, the first row of data for Chicago-O'Hare in the 2023 table has a partition key of KORD and a row key of 20230101-0051.

Climate records use a row key of "Climate-" and the date. So yesterday's climate data for Chicago-O'Hare has a partition key of KORD and a row key of "Climate-20230415". Easy to remember, and easy to construct queries.

To that end, the original (.NET 6) code looked like this:

var query = 
	from entity in table.CreateQuery<ClimateRecordTableServiceEntity>()
	where entity.PartitionKey == locationId
		&& string.Compare(entity.RowKey, lowerRowKey, StringComparison.InvariantCultureIgnoreCase) >= 0
		&& string.Compare(entity.RowKey, upperRowKey, StringComparison.InvariantCultureIgnoreCase) <= 0
	select entity;

When I upgraded to .NET 7, I naïvely just changed the first line, to this:

var query = 
	from entity in table.Query<ClimateRecordTableServiceEntity>()
	where entity.PartitionKey == locationId
		&& string.Compare(entity.RowKey, lowerRowKey, StringComparison.InvariantCultureIgnoreCase) >= 0
		&& string.Compare(entity.RowKey, upperRowKey, StringComparison.InvariantCultureIgnoreCase) <= 0
	select entity;

When confronted with a 30-day query, though, it spun off into the abyss and crashed the whole app.

The correct code looks like this:

var query = table.QueryAsync<ClimateRecordTableServiceEntity>(entity =>
	entity.PartitionKey == locationId
		&& string.Compare(entity.RowKey, lowerRowKey, StringComparison.InvariantCultureIgnoreCase) >= 0
		&& string.Compare(entity.RowKey, upperRowKey, StringComparison.InvariantCultureIgnoreCase) <= 0);

See, now the filter part of the query goes inside the method call. (There's an extra step in reading the async results back, too.)

So the effect of the naïve fix was to hit the table 30 times getting back the entire partition each time. Remember that all of the weather reports go into the table? So, yeah, the 2023 table already has something like 7.5 million rows, or about 2,500 in each partition. So it tried to read 75,000 rows just to bring back 30. Oopsi.

I'm deploying the fix now.

New Weather Now build

It took a few weeks at odd hours, but I have finally deployed the latest version of Weather Now (5.0.8507). I didn't update anything visual, but all the plumbing got a refresh. It's now running in .NET 7 (until November, when .NET 8 comes out), and I did a top-to-bottom review of its asynchronous code.

The app now runs noticeably faster, and I believe the corrections to the async bits will cure the nagging (but invisible) problem of thread exhaustion that happened from time to time.

Now I can start making some other fixes and adding some long-missing features. Really, I just needed to get back into the swing of it.

Asyncing feeling

I spent all day updating my real job's software to .NET 7, and to predominantly asynchronous operation throughout. Now I have four stubbornly failing unit tests that lead me to suspect I got something wrong in the async timing somewhere. It's four out of 507, so most of today's work went fine.

Meanwhile, the following stories have backed up:

Finally, a very rich person is very annoyed after his or her private jet got stuck in the mud at Aspen's airport. It seems the guy sent to pull it out of the mud maybe needed another lesson on how planes work, because he managed to snap the nose gear right off the $3.5 million airplane. Oopsi. (There's video!)

Twitter's long slide into irrelevance

I woke up this morning to about 450 error messages because our team's Twitter account got suspended at midnight UTC—that is, at 7pm last evening. No one knew about it because we never considered Twitter errors critical enough to keep them in our inboxes; they all go to an Outlook subfolder. Apparently, Twitter finally decided that our 15-minutes-apart API calls violated a policy, but we never got informed that this would happen or that we needed to correct something.

As near as I can figure out, Twitter's new pricing structure gives developers access to post 1,500 Tweets per month for free, but doesn't allow searches. Our app doesn't post, it only reads Tweets. The "Basic" level costs $100 and allows reading 10,000 Tweets per month. Anything more than that and you have to apply for an Enterprise license—and they don't disclose pricing information online.

Our app read about 12,000 Tweets per hour because each API call brought back about 2,000 Tweets at a time. And as far as we knew, that was what we signed up for. We even have code that checks whether we're approaching the API rate limit so we can pause the API calls.

All of this happened on the same day that Twitter decided National Public Radio is "state-affiliated media," i.e., in the same category as TASS and North Korea's "news" channel. NPR is not amused:

NPR operates independently of the U.S. government. And while federal money is important to the overall public media system, NPR gets less than 1% of its annual budget, on average, from federal sources.

Noting the millions of listeners who support and rely upon NPR for "independent, fact-based journalism," NPR CEO John Lansing stated, "NPR stands for freedom of speech and holding the powerful accountable. It is unacceptable for Twitter to label us this way. A vigorous, vibrant free press is essential to the health of our democracy."

NPR officials have asked Twitter to remove the label. They initially assumed it was applied by mistake, NPR spokesperson Isabel Lara said. "We were not warned. It happened quite suddenly last night," Lara said.

In response to an NPR email for this story seeking comment and requesting details about what in particular might have led to the new designation, the company's press account auto-replied with a poop emoji — a message it has been sending to journalists for weeks.

Mastadon user Rod Hilton posted this in December, which perfectly captures  the value of Twitter's infantile owner:

He talked about electric cars. I don't know anything about cars, so when people said he was a genius I figured he must be a genius.

Then he talked about rockets. I don't know anything about rockets, so when people said he was a genius I figured he must be a genius.

Now he talks about software. I happen to know a lot about software & Elon Musk is saying the stupidest shit I've ever heard anyone say, so when people say he's a genius I figure I should stay the hell away from his cars and rockets.

I figure, I'll keep Twitter as long as some of the people I like keep posting on it, but I know the app will eventually fail. It's a little annoying that our research at work has to stop, because now I have to build an API adapter for a new app.

I can't help but compare Musk to Eddie Lampert, the guy who destroyed the department store Sears. I despise sociopaths like Lampert, but at least Lampert had a definable business strategy and extracted value from tearing the brand apart. Musk really isn't all that smart, and Twitter isn't all that valuable. "Say what you want about the tenets of National Socialism, Dude. At least it's an ethos."

Lunchtime links

Once again, I have too much to read:

Finally, it was 20 years ago tonight that Chicago Mayor Richard M. Daley had city workers vandalize Meigs Field so that he could sell the land to his pals. The Tribune has a photo history.

After "calm" discussion, Lebanon pirouettes on daylight saving time

The Lebanese Government has capitulated after the massive outcry and international ridicule that followed their asinine decision to postpone daylight saving time on two days' notice:

Translation:

Mikati: I decided to invite the Council of Ministers to present the above. The discussion was calm and it was decided to adopt daylight saving time, starting from Wednesday-Thursday night...

Uh huh. Perhaps that's because most large institutions, including airlines and banks, told Mikati where he could stuff his decision?

Lebanon's incompetent government

Lebanon has one of the most chaotic political systems in the world. The previous government presided over a massive ammonium nitrate explosion they could have prevented had any one person in government taken responsibility for removing a derelict Russian freighter.

Once again, the Lebanese government has displayed head-shaking incompetence, this time on what seems like a minor matter but could lead to more religious unrest as hot weather combines with people not eating or drinking water during the day. Always a good combination.

So what did the good burghers of Beirut do this time? They decided on Thursday not to change to daylight saving time this weekend, most likely so that people can make Iftar "an hour earlier." Of course, changing the clock time of sunset doesn't actually change the duration of daylight; Earth spins on its axis all the same, indifferent to how we measure it. So observant Muslims in Lebanon will still fast for a little over 12 hours today, just as they did yesterday.

I found out about this idiocy right away from the Time Zone Committee email list, but some Lebanese just found out about it this morning. It hasn't gone well:

Daylight Saving will be introduced from midnight on April 20 rather than from midnight on March 25.

No official explanation has been given for the move although local media suggested it was introduced to coincide with Ramadan.

But the decision is facing widespread revolt, with two TV channels going ahead with the clock changes in protest.

MTV Lebanon and LBCI Lebanon say they will refuse to cooperate with the adjustment, announcing they will go ahead with switching to Daylight Saving Time on Saturday.

CNN also reports, "Adding to the confusion, the government is yet to say whether it has informed officials responsible for synchronizing times on mobile phones, laptops and other electronic devices of the change." This is true; the Government of Lebanon has not officially informed IANA of the change. Fortunately—or unfortunately—for them, someone sent us a link to a reliable Tweet, so we went ahead with a patch, and some guidance on how to use the new rule by changing your phone to the Libyan time zone until April 20th.

The problem with authoritarian governments, of course, is that they believe they control everything within their domains, without having the first clue how things actually get done. Add some religion and it gets even stupider.

Democracy is messy; democracy is frustrating; but democracy doesn't usually get such basic technical decisions so maddeningly wrong.

Layout frustrations

I'm arguing with the Blazorise framework right now because their documentation on how to make a layout work doesn't actually work. Because this requires repeated build/test cycles, I have almost no time to read all of this:

Finally, a group of Chicago aldermen have proposed that the city clear sidewalks of snow and ice when property owners don't. Apparently the $500 fines, which don't happen often, don't work often either.

First sunny day since I returned

We had four completely-overcast days in a row, including one with some blowing snow, so I'm happy today has been completely clear. Tomorrow might even get above 10°C—which would at least get into normal March temperatures. This whole winter has been weird, as the next few will likely be until temperature increases start leveling out.

In other news:

Finally, Bruce Schneier and Nathan Sanders explain how AI could write our laws in the future.

Following up on a few things

Perhaps the first day of spring brings encourages some spring cleaning? Or at least, revisiting stories of the recent and more distant past:

  • The Navy has revisited how it names ships, deciding that naming United States vessels after events or people from a failed rebellion doesn't quite work. As a consequence, the guided missile cruiser USS Chancellorsville (CG-62, named after a Confederate victory) will become the USS Robert Smalls, named after the former slave who stole the CSS Planter right from Charleston Harbor in 1862.
  • Author John Scalzi revisited whether to stay on Twitter, given its "hot racist right-wing trash" owner, and decides why not? It's not like Musk will ever benefit financially from the app.
  • Charles Blow revisited the (long overdue) defenestration of cartoonist Scott Adams, deciding it doesn't matter whether Adams was lazy or stupid, throwing him out the window was appropriate.
  • Illinois Attorney General Kwame Raoul revisited the Equal Rights Amendment, but the DC Circuit Court of Appeals decided yesterday not to.
  • WBEZ revisited the only other two Chicago Mayors who lost their re-election bids in the past century, Michael Bilandic and Jane Byrne.
  • A group of US intelligence agencies revisited Havana Syndrome, without finding sufficient evidence to blame either an adversary government or an energy weapon.

Finally, here's a delightful clip of US Representative Jamie Raskin (D-Md.) patiently explaining to Rep. Lauren Boebert (R-Colo.) and her banana-republican party the difference between an adjective and a noun: