|March 8th, 2018|
The web started off as just HTTP. This allowed for an enormous amount of things, but online shopping wasn't one of them. The problem was, sending credit card numbers over HTTP opened them up to theft: anyone between you and the server could keep a copy of your card information. Netscape saw this as an opportunity, if they could make a safe way to do this, using their browser and server. This would be good for them and good for the success of the web as a whole.
It was initially viewed as just a thing for credit cards. For example:
Netscape charges $1,495 for an entry-level server; adding the ability to handle secure credit card transmissions bumps the price up to $5,000
—The rise of Netscape, July 1995
Later that year, when there was a vulnerability found in the protocol, reports only discussed the impact on credit cards:
The problem in the Netscape Navigator was discovered by two first-year computer science graduate students at the University of California at Berkeley. They said a criminal could break into the software's coding system in less than a minute and gain access to credit card information.
—Netscape: Internet security flaw can be fixed, September 1995
Several weeks ago Mr. Goldberg, along with another student, David Wagner, discovered a troubling flaw in the security of a Netscape Communications Corporation software program designed to let people safely make credit card purchases via the World Wide Web.
—The New Watchdogs of Digital Commerce, October 1995
History from the late 1990s and early 2000s is hard to dig up, with so many dead sites and broken urls, but one place I looked to see things changing was old software manuals. Unfortunately, vendors seem to have been very wary about giving specific advice about what sort of pages needed HTTPS. Instead, we get things like mailing list administration software saying:
If security is a concern, consider securing access to the LISTSERV Maestro servers with encrypted communication, so that intruders cannot listen in on the communication between browser and server, and cannot gain knowledge about the data exchanged or spy out passwords.
—Securing Access With SSL, May 2002
By 2004 HTTPS was being described as "secur[ing] web sites for online banking, stock trading and retailing, and users were generally being taught that they should expect to see https:// for their bank or when buying things, but not most other places. For example, here's some advice from 2005 about forums that have logins but don't support HTTPS:
Most message boards and other sites that have membership options do not handle your login information through a secure layer, which means it is unencrypted, and anyone who manages to intercept data packets can read it.
Does this mean you should never use sites like this? No. What it does mean is, you should have a password for unsecure sites which is different from passwords you use for banking and credit card transactions, or anything else which requires security.
—Should I provide passwords at unsecure sites?, May 2005
It seems like around this time it became common practice to use HTTPS for logins even if the rest of the site was HTTP-only. For example, in 2006 the Wikipedia page for HTTPS was was edited to add that in addition to "payment transactions" HTTPS was also commonly used for "corporate logons".
Then we had a step backwards: sites realized they could put login forms on their home pages without serving the home page as HTTPS, by setting the form to POST to an HTTPS endpoint:
After years of training customers to trust only SSL-enabled sites, banks are shifting their online banking logins to the unencrypted home pages of their websites. Although the data is encrypted once the user hits the "Sign In" button, the practice runs counter to years of customer conditioning, as well as the goals of the browser makers. Three of the five largest U.S. banks now display login forms on non-SSL home pages, including Bank of America, Wachovia and Chase, as well as financial services giant American Express.
Mindful of this, many of the banks using homepage logins include a link to security information. "You may notice when you are on our home page that some familiar indicators do not appear in your browser to confirm the entire page is secure," Bank of America notes in its security note, accessed by clicking an icon on the login form. "Those indicators include the small 'lock' icon in the bottom right corner of the browser frame and the 's' in the Web address bar (for example, 'https'). To provide the fastest access to our home page for all of our millions of customers and other visitors, we have made signing in to Online Banking secure without making the entire page secure. Please be assured that your ID and passcode are secure and that only Bank of America has access to them."
—Banks Shifting Logins to Non-SSL Pages
As Microsoft pointed out at the time, this isn't secure against active attacks since the login page could be modified in transit.
This was still common in 2009:
Some major websites--such as Facebook, Vox and others (but not Metafilter!)--have users enter username & password on an http page rather than an https page. Is this as secure as logging in via https:// and if so, why?
—Are login boxes on HTTP pages secure?, December 2009
Around this time Gmail introduced an option to always use HTTPS for your account:
We use https to protect your password every time you log into Gmail, but we don't use https once you're in your mail unless you ask for it (by visiting https://mail.google.com rather than http://mail.google.com). Why not? Because the downside is that https can make your mail slower. Your computer has to do extra work to decrypt all that data, and encrypted data doesn't travel across the internet as efficiently as unencrypted data. That's why we leave the choice up to you.
We've added an option to Settings to always use https. If you don't regularly log in via unencrypted wireless connections at coffee shops or airports or college dorms, then you might not need this additional layer of security. But if you want to always use https, then this setting makes it super easy.
—Making Security Easier, July 2008
[Disclosure: I work for Google, though this post isn't a work project.]
And they made HTTPS the default a year and a half later:
If you trust the security of your network and don't want default https turned on for performance reasons, you can turn it off at any time by choosing "Don't always use https" from the Settings menu. Gmail will still always encrypt the login page to protect your password. Google Apps users whose admins have not already defaulted their entire domains to https will have the same option.
—Default https access for Gmail, January 2010
In 2010 they also added HTTPS support for search:
Years ago Google added SSL encryption to products ranging from Gmail to Google Docs and others, and we continue to enable encryption on more services. Like banking and e-commerce sites, Google's encryption extends beyond login passwords to the entire service. This session-wide encryption is a significant privacy advantage over systems that only encrypt login pages and credit card information.
when you search using SSL, you won't see links to offerings like Image Search and Maps that, for the most part, don't support SSL at this time. Also, since SSL connections require additional time to set up the encryption between your browser and the remote web server, your experience with search over SSL might be slightly slower than your regular Google search experience.
—Search more securely with encrypted Google web search, May 2010
Deployments where the site supported HTTPS but wouldn't serve it to you unless you explicitly chose to go there had been around since at least the 2004 launch of Gmail, but this was a big one and prompted the creation of the HTTPS Everywhere browser extension:
Today EFF and the Tor Project are launching a public beta of a new Firefox extension called HTTPS Everywhere. This Firefox extension was inspired by the launch of Google's encrypted search option. We wanted a way to ensure that every search our browsers sent was encrypted. At the same time, we were also able to encrypt most or all of the browser's communications with some other sites: [list of sites]
—Encrypt the Web with the HTTPS Everywhere Firefox Extension, June 2010
This extension kept a list of sites that provided HTTPS versions, along with regexps to go from an HTTP url to an HTTPS one. This was not always as simple as adding an 's' to the protocol, since many sites used to do things like having http://en.wikipedia.org for HTTP but https://secure.wikimedia.org/wikipedia/en for HTTPS Their FAQ describes their thinking:
Q. What if the site doesn't support HTTPS, or only supports it for some activities, like entering credit card information?
A. You could try to contact the site and point out that using HTTPS for all site features is an increasingly common practice nowadays and protects users (and sites) against a variety of Internet attacks. For instance, it defends against the ability of other people on a wifi network to spy on your use of the site or even take over your account. You can also point out that credit card numbers aren't the only information you consider private or sensitive.
Sites like Google, Twitter, and Facebook now support HTTPS for non-financial information—for general privacy and security reasons.
— HTTPS Everywhere FAQ, June 2010
While some users installed the extension, most, as with all things you have to go out of your way to enable, did not. For example, I don't think I did.
Over the course of the decade people had been shifting from mostly wired internet connections to mostly wireless ones. It had become common for people to connect to WiFi in public places like coffee shops, and this changed the effective security level of HTTP. Instead of an attacker needing to subvert your ISP or an intermediate network, they just needed to listen to what was being sent in the open over WiFi. Using HTTPS only for login but then sending authentication cookies in the clear on each request had never been all that secure, but now it was easily exploitable. Still, users and companies mostly didn't worry about it.
Then in October 2010 Firesheep came out. This was an active exploit, packaged as an easy-to-use Firefox extension. Anyone could go down to the local coffee shop, open up Firesheep, sniff people's cookies, and impersonate people on Facebook, Twitter, etc. The author of Firesheep wrote:
The only effective fix for [session hijacking] is full end-to-end encryption, known on the web as HTTPS or SSL. Facebook is constantly rolling out new "privacy" features in an endless attempt to quell the screams of unhappy users, but what's the point when someone can just take over an account entirely? Twitter forced all third party developers to use OAuth then immediately released (and promoted) a new version of their insecure website. When it comes to user privacy, SSL is the elephant in the room.
—Firesheep, October 2010
Companies hurried to at least offer an HTTPS option, and Facebook announced an opt-in setting in January 2011, along with a goal of having HTTPS as the default at some point:
Starting today we'll provide you with the ability to experience Facebook entirely over HTTPS. You should consider enabling this option if you frequently use Facebook from public Internet access points found at coffee shops, airports, libraries or schools.
There are a few things you should keep in mind before deciding to enable HTTPS. Encrypted pages take longer to load, so you may notice that Facebook is slower using HTTPS. In addition, some Facebook features, including many third-party applications, are not currently supported in HTTPS. We'll be working hard to resolve these remaining issues. We are rolling this out slowly over the next few weeks, but you will be able to turn this feature on in your Account Settings soon. We hope to offer HTTPS as a default whenever you are using Facebook sometime in the future.
—A Continued Commitment to Security, January 2011
Three months later, Twitter added support:
we are adding a user setting that lets you always use HTTPS when accessing Twitter.com.
This will improve the security of your account and better protect your information if you're using Twitter over an unsecured Internet connection, like a public WiFi network, where someone may be able to eavesdrop on your site activity. In the future, we hope to make HTTPS the default setting.
—Making Twitter more secure: HTTPS, March 2011
Google was already using HTTPS by default for Gmail, and offered an option to use search over HTTPS, but search was still unencrypted by default. In October they rolled out HTTPS by default for logged in users:
As search becomes an increasingly customized experience, we recognize the growing importance of protecting the personalized search results we deliver. As a result, we're enhancing our default search experience for signed-in users. Over the next few weeks, many of you will find yourselves redirected to https://www.google.com (note the extra "s") when you're signed in to your Google Account. This change encrypts your search queries and Google's results page. This is especially important when you're using an unsecured Internet connection, such as a WiFi hotspot in an Internet cafe.
—Making search more secure, October 2011
Other providers moved slower. For example, Yahoo Mail didn't even offer the option to always use HTTPS until late 2012, when they apparently introduced it without an announcement. Small companies were generally even slower to add support, unless they were explicitly privacy or security focused.
In June 2013 Edward Snowden revealed how much spying the US had been doing, and how pretty much all HTTP traffic was being monitored. It looks to me like these disclosures really changed how technical people were thinking about HTTPS. Before the idea was that you should use HTTPS just for sensitive things (purchases, financial stuff, logged in sessions) but most things could stay in the clear (news sites, blogs, reference), while after the disclosures there was much more support for the idea that eventually everything should be over HTTPS.
In the short term, companies that hadn't been using HTTPS consistently for sensitive things scrambled to roll it out:
We now use https by default for all Facebook users. This feature, which we first introduced as an option two years ago, means that your browser is told to communicate with Facebook using a secure connection, as indicated by the "https" rather than "http"
—Secure Browsing By Default, July 2013
Our current architecture cannot handle HTTPS by default, but we've been incrementally making changes to make it possible. Since we appear to be specifically targeted by XKeyscore, we'll be speeding up these efforts.And then followed up with HTTPS by default for logged in users later that month.
—The future of HTTPS on Wikimedia projects, August 2013
Yahoo was again on the slower end, finally deploying it at the beginning of 2014:
As we promised back in October, we are now automatically encrypting all connections between our users and Yahoo Mail. Anytime you use Yahoo Mail - whether it's on the web, mobile web, mobile apps, or via IMAP, POP or SMTP- it is 100% encrypted by default and protected with 2,048 bit certificates.By this point most of the barriers to HTTPS adoption had been resolved. You no longer needed an IP for each host , and the extra computation was too low to matter, but certificates were still annoying. For example, this was the main reason I didn't offer HTTPS on any of my (very much "long tail") sites. Let's Encrypt was founded to fix this:
—HTTPS Now Default in Yahoo Mail, January 2014
Then why don't we use TLS (the successor to SSL) everywhere? Every browser in every device supports it. Every server in every data center supports it. Why don't we just flip the switch?
The challenge is server certificates. The anchor for any TLS-protected communication is a public-key certificate which demonstrates that the server you're actually talking to is the server you intended to talk to. For many server operators, getting even a basic server certificate is just too much of a hassle. The application process can be confusing. It usually costs money. It's tricky to install correctly. It's a pain to update.
Let's Encrypt is a new free certificate authority, built on a foundation of cooperation and openness, that lets everyone be up and running with basic server certificates for their domains through a simple one-click process.
—Let's Encrypt: Delivering SSL/TLS Everywhere, November 2014
At this point browers could start pushing pretty heavily towards HTTPS. As they introduced new features, especially ones with any security impact, they made them only available to HTTPS pages. (service workers, getUserMedia, Prefer Secure Origins For Powerful New Features) In September 2016 Chrome announced plans to eventually mark all HTTP sites as insecure:
Historically, Chrome has not explicitly labelled HTTP connections as non-secure. Beginning in January 2017 (Chrome 56), we'll mark HTTP pages that collect passwords or credit cards as non-secure, as part of a long-term plan to mark all HTTP sites as non-secure.
—Moving towards a more secure web, September 2016
And this is still on track:
For the past several years, we've moved toward a more secure web by strongly advocating that sites adopt HTTPS encryption. And within the last year, we've also helped users understand that HTTP sites are not secure by gradually marking a larger subset of HTTP pages as "not secure". Beginning in July 2018 with the release of Chrome 68, Chrome will mark all HTTP sites as "not secure".
—A secure web is here to stay, February 2018
You can see the overall rise of HTTPS from Google's HTTPS Transparency Report:
You can also see it from Firefox data, via Let's Encrypt stats:
We're not yet to where there is only https://, but it looks like we'll be there within a few years.
 In 2003 RFC 3456: Transport Layer Security (TLS) Extensions came out, which defined Server Name Indication (SNI):
TLS does not provide a mechanism for a client to tell a server the name of the server it is contacting. It may be desirable for clients to provide this information to facilitate secure connections to servers that host multiple 'virtual' servers at a single underlying network address.
—RFC 3456, June 2003.
For example, I host jefftk.com, trycontra.com, lilywise.com, and several other sites all on the same server with a single IP address. When a request comes in, it specifies which of these it wants with the Host: header:
GET /index HTTP/1.1 Host: www.jefftk.com ...Before SNI, if I had been trying to host all of these on HTTPS at one IP, the conversation would have started with a request for a certificate. But how would I know which certificate to provide? If they're trying to visit jefftk.com they want Cert A, while for trycontra.com they want Cert B, etc. At the time you could work around this by using different IP addresses for the different sites, and then you know a request on IP A should get Cert A, while IP B should get Cert B. This was expensive and awkward, however, and limited the deployment of HTTPS.
(There was another way to do it, subjectAltName to make a single certificate that was good for multiple domains. This would have been fine for my sites, and actually is still how I handle certs, but it doesn't work well for hosting providers or anyone else with a lot of domains that change often.)
With SNI, the browser includes the domain name of the site it's trying to visit when it first starts setting up an SSL connection. This lets the server pick the right certificate to serve, without relying on the multiple IP addresses hack.
On the other hand, as long as any browsers you want to support don't handle SNI yet you can't switch over to it. This was a very long transition. Since SNI was handled by the SSL library, support for SNI was usually dependent on the operating system version. Windows XP and Android Gingerbread were the last two widely used OSes to not support SNI, and they stuck around for a long time. For example, in July 2013 Gingerbread and earlier were still at 41%, in July 2015 they were at 6%, and they didn't drop below 1% until June 2017. (See pretty charts.) if you specifically visited https://gmail.com.