blufive | Entries tagged with web

Far-too-many-years ago, at either a pub or a party, I was ranting about how annoying cookie banners were not, in fact, top-to-bottom pure evil, but were a consequence of a well-meaning-but-terribly-implemented attempt to Stop Bad Things. A (very) non-technical friend asked me to explain further, but I gave up on a verbal explanation after it became apparent that this stuff is almost fractally complex; pretty much every part of the puzzle requires significant background explanation before it makes sense (at least, way more than I could marshall in a casual conversation). I foolishly promised a written version in the near future.

This is that written version, aimed at non-technical readers. I repeat: this is (deliberately) a very, very, non-technical summary. I am skipping over so many technical details, in the interests of making this comprehensible to non-technical people and "only" ~1800 words long...

(If you understand some of those technical details, congratulations! Their omission from this essay is A Feature, Not A Bug. The footnotes dig a little deeper, but I'm trying to avoid exponential footnote growth, too)

In the beginning, the World Wide Web got invented. It was easy to hack together a basic website, and lots of people did.

Shortly afterwards, people started putting adverts on these new-fangled "web page" things. Pretty soon, it was possbile to effectively "sell" some fraction of your web page to a web advertising company, who would give you a small sum of money every time a reader saw one of their adverts on your page.

For this story, the important bit is that the "advert" bit of the web page is now controlled by the advertising company.

This means that the readers' web browsers are talking to the ad company's server, as well as the server of the webpage hosting the advert[1]. In turn, this means that the ad company's server knows which adverts it's sending to which users on which web pages.

The advertising companies soon realised that, as well as simply working out how many users saw advert-A on webpage-X, and advert-B on webpage-Y (which is necessary for their core business) they could also tell which readers were visiting both webpage-X and webpage-Y.

Multiply that up by billions of webpages (which tends to happen on the web) and each of the advertising companies had built themselves their own world-wide-web surveillance network. They could literally track which webpages were being viewed by which people, for approximately all webpages[2] and all web readers[3]. So they did just that.

That tracking is how they know that you just visited [car maker]'s website, and will now show you car adverts on every page you visit for the next three days.

Aside: This is now how those advertising companies make most of their income: by selling ad space based on the interests of who they will show it to. Want to advertise [cars|fantasy books|cat food]? We'll show your adverts only to people interested in [cars|lord of the rings|cats], and not waste time showing them to people interested in [bicycles|sci-fi|dogs].

Aside aside: this interest-targeting is a powerful (but stealthy) tool, and is being used by Bad People for Bad Things[4].

Customers started to notice adverts "following" them around the web. Technically-adept people did some quick 2+2=4 to figure out what was going on, and were Not Happy, because, y'know, Pervasive Surveillance is Bad[6].

Thus, people started making attempts to Stop This Sort Of Thing.

I'm fairly sure that someone asked the advertising companies to not do that, please. I'll leave you to imagine how successful that was.

There were also some technical attempts, but they didn't work very well, either.

The first major legal attempt to block things was the Dreaded Cookie Law[7].

(Why "cookie"? As well as the common biscuit-related meaning, the word "Cookie" is the somewhat whimsical name for a fairly fundamental bit of web technology[8] that was, at least initially, the technical linch-pin holding these surveillance networks together)

So now we come to the "terribly-implemented" bit. The core idea of the cookie law is to require consent from the end user. Consent, you say? Consent to what, exactly?

Somehow[9], this law and the surrounding guidance ended up being about "consent to use cookies ~~to conduct surveillance~~", rather than "consent ~~to use cookies~~ to conduct surveillance".

Also, the requirement to obtain this consent ended up with the webpages rather than with the adverts, forcing the implementation onto every web page publisher everywhere. (and also allowing the ad companies to apply financial pressure: no consent from your readers? less ad money for you!)

Doing the implementation really properly requires deep discussions between [people who understand all the technical details that I'm leaving out] and [people who understand all the legal details that I'm leaving out] probably resulting in them cross-teaching each other the whole mess. A proper outcome may involve hitting the bottom line of the company running the pages, too (because it may mean reducing advertising income, or compromising the ability to track how effective their own advertising is, or both).

At which point you have a lengthy three-way discussion between the legal/technical/commercial arms of this hypothetical company. Executives who want the minimum-staff-effort way to achieve minimum-legal/commercial risk will just hurry things along, and eventually someone says "[expletive] it!" and you end up with some sort of compromise, probably involving a cookie banner full of deliberately-obfuscated words and UI dark patterns, aimed at getting as many readers as possible to click the "just make this damn banner go away, dammit ~~and let them track me forever~~" button.

(and of course, most small companies and approximately all private individuals may not even have any [people who understand the details] and will probably just end up applying one of the standardized off-the-shelf tools which offer that same end result)

So you get cookie banners everywhere. And the major web advertising companies[10] are watching your every online move.

The next major legal attempt to stop this stuff was part of GDPR (and similar laws in other countries), which is attempting to go after the "surveillance" angle and the consent angle[11] but that's a whole other can of worms...

[1] This is where the term "third-party" enters the discussion. The web page owner's server is the "first-party". The user's web browser is the "second-party". The advertising company's server is the "third party".

[2] Superficially, it's limited to the web pages that show any kind of advert. But the advertising companies are cunning, and have found ways around that limitation. (log in with..., web page analytics tools, and more)

[3] Superficially, they can't attach real names to this info, either, it's all just [reader x] and [reader y]. But from many viewpoints, that's not actually much of a limitation anyway. Plus, the advertising companies are cunning, and have found ways around that limitation. Facebook's "real name" policy, for example? Or Google (via GMail) getting access to all your email?

[4] "Interest-based ad targeting". Let's think like a villian for a moment: How about you show an ad for my self-published book (about the evil conspiracy of [socialists|immigrants|boogeymen]) to people interested in [pickup trucks|looking for a job], but hide the ad from people interested in [left-wing news outlets|polital news][5]. Now, start thinking like an unscrupulous/authoritarian politician, or a PR company hired by a global megacorporation, and think of all the ~~terrifying~~ interesting ways you can use this. But that's a whole other essay.

[5] Now, you're not supposed to be able to target ads by race/religion (or by policital leaning in some parts of the world) but bad guys are cunning and found ways around that limitation. How about people interested in cycle lanes, guns, "states' rights", brexit, hip-hop/country/whatever music, or [insert job/skill here]? Or living in [specific neighbourhood/city]?

[6] Some nominally anti-authoritarian people claim to be fine with this pervasive surveillance, because it's Free Enterprise Surveillance, not Evil Government Surveillance, and therefore okay. But (can you see it coming yet?) governments are cunning, and have found ways around that limitation.

[7] IIRC, it's actually a cluster of related laws, primarily in the EU, but "skipping technical details" includes the legal details, so moving swiftly on...

[8] I'm not going to explain cookies much here. Except to say that if you turn them off completely, half the web (including most of the e-commerce) will just stop working, and That Would Be Bad. Let's just say there are good cookies, bad cookies and shades-of-grey cookies. Plus is is where that "third-party" term makes its second appearance, and that's a technicality I've now failed to completely omit so I'll stop before this footnote outweighs the rest of the essay...

[9] I'm sure this had nothing at all to do with lobbying by web advertising companies.

[10] I keep saying "the major web advertising companies" without naming them. I'm not 100% up-to-speed on the balance of power here, but AFAIK the big players are Alphabet/Google, Meta/Facebook and X/Twitter. Lesser-but-still-huge players include Microsoft, Apple and Amazon and potentially several more I'm forgetting.

Another aside: these companies have successfully injected themselves into the global advertising market as middle-men, and are hoovering up a huge slice of the money in that market. If you want to know why [newspapers/magazines/your favourite periodical publication] has no money these days, it's because these guys ate most of their advertising revenue.

[11] To paraphrase something I heard in a very different context: Submission under duress is not Consent.

If you work in the business of generating web pages to display to users, you need to know: there's going to be some major changes in the browser landscape over the next year or three.

There are several things going on out there. Any one of them could make a serious difference to the web at large - but they're all either happening right now, or imminent.

Mobile devices are going be everywhere.

I'm seeing reports[1] that at least one major chipmaker is planning to ship a system-on-a-chip imminently which they reckon will allow roughly-iPhone-3GS-equivalent handsets to sell at retail, without contract, for ~$100. Those things are going to be hitting the high street in less than 6 months.

I'm not sure I believe that precise graph of price vs. time (or even that I read it right) but I don't think they're wrong by more than a factor of two on either axis. Smartphones (more accurately, pocket frackin' computers[2]) are going to go mass market, and soon.

In 2010, smartphones sold roughly 300,000,000 units. That's about the same as the number of desktop PCs sold in the same period, apparently. This year, they're going to sell more. When those $100 beasties hit, it's going to seem like everyone's got one.

Mobile devices are not second class web citizens any more

These things have proper browsers on them. Most are webkit-based (like Chrome and Safari) so they do proper standards-based rendering, including lumps of the new CSS3 whiz-bangs. They have proper javascript engines with better performance than you may expect.

I did some ad-hoc performance testing recently. Mobile safari on iOS 4-point-(mumble) on an iPhone 3GS was about a factor 10 slower than Chrome 8 on my workstation. So: only 10 times slower than one of the fastest browsers out there running on much beefier hardware. That's comfortably in the same league as (probably even somewhat faster than) IE8 on the same workstation. It leaves IE7 in the dust.

Add those two items together (low prices + web capability) and you get: mobile devices are going to become a very popular way to access the web. This is going to happen fast - if we're lucky, we've got a year or two to prepare for it. Right now, I'm seeing mobile safari at about 1.5% to 3% of sessions on the websites I work on. That's up from basically zero 12 months ago.

All those browser support conversation, where you say things like "we'll support all the browsers over 5% market share in our stats"? They're going to get gate-crashed by mobile devices in less than 12 months' time, quite possibly less than 6. Maybe not any one browser on its own, but in aggregate, it all adds up fast.

The only thing I can see slowing this down is the mobile phone operators, and their we're-not-price-gouging-our-customers-honest-guv data payment plans. Even on that front, for $100 or so, I'd be seriously considering buying one of these beasties for use via wi-fi hotspots only (hell, that covers my house and garden, for starters - anything else is a bonus) and screw the phone people.

It's the browser wars all over again

All these new devices come with their own browsers and, unfortunately, there are loads of the buggers [3]. They're all subtly different. They're all running on different hardware, with different screen sizes. You're really, really, really, going to have to test on the real damn hardware.

The good news is that they mostly follow the standards. If you can:

persuade the photoshop weenies to give up on pixel-perfection ('cos the screen dimensions are all over the show)
do at least a bit of work to tailor the design to the realities of small screens (by dumping cosmetic fripperies and focusing on making the site suitable for whatever it's supposed to actually do)
refactor all the UI stuff to cope with touch instead of/as well as mouse interactions (which is quite a big deal, especially if the site's supposed to do any quantity of data entry)
avoid the real bleeding-edge stuff like webGL

...then you shouldn't have too much trouble keeping the browsers mostly in line.

Javascript performance has just gone up like a rocket.

IE is still the biggest desktop browser, and it sucks, especially the older versions. On the desktop, almost everyone still has to support IE6/7[4]

All the other desktop browsers are an order of magnitude faster than IE8, which is itself significantly quicker than IE7 and 6. IE9 is going to be seriously competitive with everyone else. Don't get me wrong, performance is still an issue, but the browsers are now probably less of a bottleneck than your code is.

Now, that's not bleeding-edge, but it's sure as hell quick enough to run non-trivial things at a sensible speed. Remember: there was a perfectly-playable port of Lemmings to JS done years ago, before all this optimisation occurred. Some of the stuff that people are doing out there in browser-specific-tech-demo land is amazing (like, real-time-chromakey[5], real-time hardware-accelerated 3D graphics[6])

So: You can now write real app-scale code in websites browsers. This has started to get through to people, though I still see a lot of comments like "javascript is a toy, you can't do anything serious with it" from people who really should know better, if only because it's been smacking them in the face for a year or two now.

Conclusions

Mobile devices are going to become at least a significant way of accessing the web within months. In the long run, a good chance of becoming the dominant way. That's long run in internet time - I'll go out on a limb and say: 5 years, maybe. Mobile devices are probably going to be what finally rams the stake through the hearts of IE6, 7 and maybe 8 (with Firefox and Chrome cheering from the sidelines).

The possibilities for mobile devices to run really cool things via pure web tech are much greater than most people realise - and web tech runs on all of them without having to port to a zillion different native development environments, or get sign-off from Apple before you can ship anything. [7]

The references I can be arsed finding, with added footnotes

[1] http://tech.fortune.cnn.com/2010/12/22/2011-will-be-the-year-android-explodes/

[2] Not that long ago, the state-of-the-art in phone games was "snake". An iPhone 3GS (which is now moderately old hat, tech wise) has far more computational shove than the system I used to play Half-Life all the way through. Yeah, these things happen to be phones, but, first and foremost, they're full-on pocket computers. People just haven't really cottoned on to that bit yet, because the future just went and snuck up on us.

[3] http://www.quirksmode.org/blog/archives/2009/10/there_is_no_web.html [8]

[4] On sites I interact with professionally, I'm still seeing IE6 at about 5-8% and IE7 at 12-20%. I can't quite pull the trigger yet, but IE6's days are numbered.

[5] https://developer.mozilla.org/samples/video/chroma-key/index.xhtml works in firefox 3.5 and up, allegedly. No idea about other browsers.

[6] http://webglsamples.googlecode.com/hg/aquarium/aquarium.html. You'll need a Firefox 4 beta or another WebGL-enabled browser (I believe the pre-release chrome builds can do it too, though I've not confirmed it personally)

[7] I hear the cry: "How do people make money from web apps?". That requires rather more prognosticating than I'm willing to commit to right now. One possibility is Operator Billing - it's going great guns in Africa. But that requires the mobile phone networks to be more forward-sighted than "give us $LOADSAMONEY-per-byte, and Get Orf Our ~~Land~~ Network" and "new handset is 10% better than old one, upgrade now!", so colour me somewhat skeptical for the moment.

[8] On the subject of mobile browser landscape and mobile web, just read everything PPK writes. Even if/when he's wrong (not often, on past track record) he's wrong in interesting and educational ways.

LJ have just tweaked their HTML/CSS cleaner, according to the support page. In the process, they drove cart and horses through my carefully hand-crafted S1 style here. It may take come time to sort it out. If anyone can point me at a list of what exactly this new beastie is twitchy about, it could make my life a lot easier; otherwise I'm going to have to reverse-engineer things to work out what is and isn't allowed.

While they have some legitimate concerns* I think they're currently stripping stuff that's pretty harmless. For example, how the hell can <style type="text/css"> be malicious?

Well, at least it proves that my "graceful degradation" works as intended...

*there are some downright terrifying browser-specific features out there, from the perspective of defending against cross-site scripting attacks.

[edit: there was a post on the subject on

lj_maintenance shortly after a wrote this. I think I'll wait a day or two for things to settle down before I attempt to clean up. I mean, it's not like many people read this journal in the native style, rather than via their own LJ-friends view or some other aggregator]

Dear Compuware,

Speaking as someone who writes web applications professionally, the web interface to TrackRecord is a huge, steaming pile of shite.

Let me enumerate the ways:

Accidentally clicking the incorrect option in a dropdown list results in a bug being snatched out of my hands allocated somewhere I can't edit it.
ditto if I carelessly used the mousewheel while focus is in the wrong place. This is why using "onchange" handlers to submit forms is BAAAAD, m'kay?
Once I've started an edit, I can't escape. <valleygirl>Guys, like, Transactional database access? Duh!</valleygirl>
Something resembling a useful search interface that doesn't require configuration of specific queries by a sysadmin would be cool, too.
It corrupts its own database on a daily basis (ok, that's not necessarily the web interface, but I thought I'd slip it in anyway)

Yours, etc.

Dear Employer,

Give me a real bug-tracking system to work with. If you're too terrified of (for example) bugzilla, I'll even take the old in-house character-cell-based relic instead. It may use arcane keystrokes, but it only corrupts the database once or twice a week, and the search facilities let me slice'n'dice by client, or who's dealing with it, or when it was logged, or status, and I can scroll through lists of bugs a good deal quicker than the average glacier. Oh, and once a bug is closed, it disappears unless I specifically go looking for it, rather than just cluttering the place up.

Yours, etc.

Current Music: Craig Charles Funk Show, BBC 6Music

Some official details on IE7: they've fixed PNG alpha transparency! Now, we've only got to wait until IE6 is a minority browser before using it freely...