kottke.org posts about search
Ok, after a week of testing, I’ve opened up KDO search to everyone. You can read more about it in my post from last week:
For now, the search only looks at posts from 2014 to the present. You can search by keyword, describe what you remember (“the guy who walks in Japan”), or paste a link to find posts that link to it. Search is fuzzy-ish but it’s not an AI bot or anything like that so if you ramble on with a 25-word query, it’s probably not going to work that well. You can sort by best match or newest, search all posts or just Quick Links or main posts, and filter by time (past month, past 3 months, etc.)
Launching that member preview was a little deflating tbh — “no plan survives first contact with the user”. It was clear that search was not as good as I wanted it to be — even I started noticing it despite my excitement. I spent quite a bit of time last weekend trying to tune the search and it’s much improved. I’ve been using it quite a bit this week and am loving it. There’s so much good stuff in the archives (and I can say that because if it’s more than 3-4 years old, I don’t even remember writing it! 🤷♂️) and this does a great job of surfacing that stuff.
Speaking of surfacing more posts from the archive, one of the coolest things about the way the search works is that you get a “more like this post” capability more or less for free. If you open recent posts like the video of three humpback whales breaching, the composite photo of 14,000 satellite trails, this Ted Chiang interview, or even an older post like Sesame Street puppeteers on how they play their characters and click on the “Related posts” tab below the post, you should see links 1-5 posts that you may want to read next.
Like the search, it’s not perfect and not every post will have related links, but I keep turning up good posts I’d completely forgotten about. Anyway, related posts are available for everyone on post-2014 posts. Post post post post. Poooooost. Ok, getting punchy. Have a good weekend everyone.
After years in the making (or more accurately, years in the wanting), I finally figured out how I wanted to do search for KDO posts. I’ve been tinkering with this specific implementation since the beginning of the summer and it finally all came together. If you’re a member, please help me to kick the tires on this for a few days. (That link, as well as the search links below, will 404 for non-members for now.)
Hopefully it’s pretty close and then I can open it up to everyone. I’ve been using it for the past week or two and it’s been really helpful so far, especially in surfacing older posts about topics I’m currently writing about or generally interested in. The tag pages are good for that too, but search can help hone in on specifics — like posts I’ve made about fascism or posts that link to McSweeney’s. Search also pulls in results from Quick Links, which aren’t included on tag pages. The rankings still need some work here and there (it’s not magic), but your searches will help calibrate things.
For now, the search only looks at posts from 2014 to the present. You can search by keyword, describe what you remember (“the guy who walks in Japan”), or paste a link to find posts that link to it. Search is fuzzy-ish but it’s not an AI bot or anything like that so if you ramble on with a 25-word query, it’s probably not going to work that well. You can sort by best match or newest, search all posts or just Quick Links or main posts, and filter by time (past month, past 3 months, etc.)
I’ll share more about how search works in a subsequent post. For now, kick the tires and let me know what you think in the comments or via email. As always, bug reports, comments, and suggestions are welcome.
Megan Gray on a disturbing piece of information that was revealed in the antitrust case against Google: their search engine replaces some search queries with others that generate more commercial results (and therefore more money for the company). Here’s how it works:
Google likely alters queries billions of times a day in trillions of different variations. Here’s how it works. Say you search for “children’s clothing.” Google converts it, without your knowledge, to a search for “NIKOLAI-brand kidswear,” making a behind-the-scenes substitution of your actual query with a different query that just happens to generate more money for the company, and will generate results you weren’t searching for at all. It’s not possible for you to opt out of the substitution. If you don’t get the results you want, and you try to refine your query, you are wasting your time. This is a twisted shopping mall you can’t escape.
Yuuuck. I think it might be time to switch away from Google search — its results have been getting worse for years and it seems like the company doesn’t care too much about fixing it. I’ve been hearing good things about Kagi and there’s always DuckDuckGo.
Update: Several people wrote in noting that Gray’s article was an opinion piece, not reported, and that there was no corroboration of her claim of Google’s query switcheroo. For their part, Google denies the claim: Per Platformer:
Google does not delete queries and replace them with ones that monetize better as the opinion piece suggests, and the organic results you see in Search are not affected by our ads systems.
(thx, andy)
Update: Wired has removed the story from their website:
After careful review of the op-ed, “How Google Alters Search Queries to Get at Your Wallet,” and relevant material provided to us following its publication, WIRED editorial leadership has determined that the story does not meet our editorial standards. It has been removed.
The Internet Archive has a copy of the original piece. Charlie Warzel has more in a report from the Atlantic. (thx, andy)
Flim is a movie search engine currently in beta that returns screenshots from movies based on keywords like “clock” or “tree”. Like so:
You can filter results by things like genre, year, and film ratio. You can search by color and within movies, e.g. “tuxedo” in Titanic:
I would love for the screenshot detail pages to include timecodes — it would make this an amazing tool for creating supercuts, film analysis videos, and other sorts of media. Imagine how much easier Christian Marclay’s job would have been with “clock” and “watch” searches on Flim. (via waxy)
UK Prime Minister Boris Johnson appears to be strategically using particular words and phrases in speeches and appearances as SEO bait to bury unfavorable news about himself in Google’s search results. My pal Matt Webb has collected three examples of this devious practice over the past month.
Not only has Boris used his infamous ‘dead cat strategy’ to move the conversation away from him and Carrie Symonds and his plans for Brexit, he’s managed to push down his past mistakes on Google, too — making it more difficult for people to get a quick snapshot of relevant information. He’s not just controlling the narrative here — he’s practically rewriting it. And judged by the standards of an SEO campaign, it’s hard to describe it as anything other than a resounding success.
In the latest instance, Johnson used the phrase “model of restraint” in a TV appearance, which then came up in search results for “boris johnson model” instead of articles about the allegations that he’d had a sexual relationship with a former model whose business he funneled money & favors to while mayor of London.
Wired has more information on the PM’s potential SEO scam.
His speech in front of the police was meant to distract from reports that the police were called to the flat he shared with girlfriend Carrie Symonds following an alleged domestic dispute, while the kipper incident was meant to downplay connections with UKIP (whose supporters are called kippers). The claim about painting buses, finally, was supposedly intended to reframe search results about the contentious claim that the UK sends £350 million to Europe branded on the side of the Brexit campaign bus.
“It’s a really simple way of thinking about it, but at the end of the day it’s what a lot of SEO experts want to achieve,” says Jess Melia of Parallax, a Leeds-based company that identified the theory with Johnson’s claim to paint model buses.
“With the amount of press he’s got going on around him, it’s not beyond the realm of possibility that someone on his team is saying: ‘Just go and talk about something else and this is the word I want you to use’,” says Melia.
Update: If you didn’t click through to the Wired article by Chris Stokel-Walker, the piece presents a number of reasons why Johnson’s supposed SEO trickery might not work:
For one thing, Google search results are weighted towards behavioural factors and sentiment of those searching for terms — which would mean that such a strategy of polishing search results would be shortsighted. The individual nuances of each user are reflected in the search results they see, and the search results are constantly updated.
“What we search for influences what we find,” says Rodgers. “Not all search results are the same. That front page of Google, depending on what I’ve searched for in the past. It’s very hard to game that organic search.”
Current searches for the terms in question show that any effect was indeed short-lived. On Twitter, Stokel-Walker says that “No, Boris Johnson isn’t seeding stories with odd keywords to reduce the number of embarrassing stories about him in Google search results” (and calls those who believe Johnson is doing so “conspiracy theorists” (perhaps in mock frustration)) but the piece itself doesn’t provide its readers such a definitive answer,1 instead offering something closer to “some experts say he probably isn’t deliberately seeding search keywords and others disagree, but even if he is, it is unlikely to work as a long-term strategy”. Since we don’t really know — and won’t, unless some Johnson staffer or PR agency fesses up to it — that seems like an entirely reasonable conclusion for now.
This season on Halt and Catch Fire, competing teams made up of the main characters are building web apps. One team is making a hand-curated directory called Comet and the other is building a search engine with a unique algorithm called Rover. It’s Yahoo vs. Google, more or less. AMC has put up the sites as they appear on the show up on the web: Comet and Rover. They also put up the Cameron Howe fan page seen in the most recent episode, The Howe of It All (that’s a perfectly anachronistic name). See also Yahoo’s website circa 1994 when it was still hosted on a server at Stanford.
A character in the most recent episode also mentioned Justin Hall, creator of the early web’s most well-known personal site, Justin’s Links from the Underground.
Anyone know where I can get a Comet t-shirt? (I already have a Cardiff Electric shirt. (No, really.)) Oh, maybe this?
According to Google Trends, search traffic about the upcoming solar eclipse mirrors the path of totality. And according to XKCD, pre-eclipse search traffic for “eclipse” is outpacing pre-election search traffic for “election”.
For his book Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are, Seth Stephens-Davidowitz combed through data on Google Trends for five years, looking for data on searches that Google users “don’t tell to possibly anybody else, things they might not tell to family members, friends, anonymous surveys, or doctors”.
According to a recent interview with Stephens-Davidowitz, right now the data is showing an increase in search queries on how to perform abortions at home and, no surprise, the activity is highest in parts of the country where access to abortion is most difficult.
I’m pretty convinced that the United States has a self-induced abortion crisis right now based on the volume of search inquiries. I was blown away by how frequently people are searching for ways to do abortions themselves now. These searches are concentrated in parts of the country where it’s hard to get an abortion and they rose substantially when it became harder to get an abortion. They’re also, I calculate, missing pregnancies in these states that aren’t showing up in either abortion or birth rates.
That’s pretty disturbing and I think isn’t really being talked about. But I think, based on the data, it’s clearly going on.
Morbotron is a screencap search engine for Futurama. The cap above is perhaps the most popular use case. It also does animated GIFs. See also Frinkiac, the Simpsons screencap search engine.
Terrapattern is a search engine for satellite images. You click on a specific feature of interest on a map and the site returns results that match it. For instance, here are the locations of solar panels in NYC.
You can also use Terrapattern to find school bus depots, fracking wells, Air Force bombers, baseball diamonds, train tracks, and much more.
There are only four cities currently represented (Pittsburgh, New York, San Francisco, and Detroit) but this is already super cool to play around with. (via @genmon)
Frinkiac searches through the subtitles from every episode of The Simpsons (in the first 15 seasons) and returns screencaps of all the times when the search term was used. For example, inanimate:
(via @emunn)
Update: Hi diddley ho! The Frinkiac now does animated GIFs. Like this one. (via @buzz)
This map was compiled using the autocomplete results for “how much does a * cost” for every country in the world.
Some notable desires: Mexican tummy tucks, Brazilian prostitutes, Albanian nose jobs, Russian MiGs, Lebanese PS3s, and Japanese watermelons.
See also the desire map of the US.
In Alaska, people search for the cost of a gallon of milk. In Alabama and Florida, people search for the cost of abortions. In other states, vasectomies, facelifts, and taxis are popular searches. The map was compiled using the autocomplete results for “how much does a * cost”… for each of the 50 states. (via mr)
Yobi3D is a search engine for finding 3D objects. Here’s a search for “horses”:
This is pretty neat…all the objects are zoomable and rotatable in the browser. (via prosthetic knowledge)
Lantern is a search engine for the books, periodicals, and catalogs contained in the Media History Digital Library. If you are a fan or student of pre-1970s American film and broadcasting, this looks like a goldmine. Here are some of the periodical titles and the years available:
Variety 1905-1926
Photoplay 1914-1943
Movie Classic 1931-1937
Home Movies and Home Talkies 1932-1934
Talking Machine World 1921-1928
(via candler blog)
Older posts
Socials & More