Bing (and DuckDuckGo) shadow banned my website
4 min read (1100 words)
My domain was shadow banned by Bing and DuckDuckGo last year, and I don’t know why. Since then, my root domain (rubenwardy.com) has been unbanned but my blog remains banned. The event also negatively impacted the search placement of my root domain; another site that reuploaded some of my content is appearing as the first result when searching for it.
My domain was shadow banned by Bing and DuckDuckGo last year, and I don’t know why. Since then, my root domain (rubenwardy.com) has been unbanned but my blog remains banned. The event also negatively impacted the search placement of my root domain; another site that reuploaded some of my content is appearing as the first result when searching for it.
Table of contents
Events
June 2022: Discovered my domain was banned
Last year, I received a message from a user saying that my website wasn’t appearing on DuckDuckGo. Looking into it, I discovered that DuckDuckGo uses Bing and my website wasn’t appearing there either.
The first thing I did was sign all my domains up to Bing Webmaster. The dashboards were empty, and using the URL inspection tool resulted in the following generic error:
I’ve read through the Bing Webmaster Guidelines and I’m certain that my website follows them.
rubenwardy.com hosts my portfolio, blog, and several open-source projects - including the book I wrote about creating mods for Minetest. At this point last year, my best guess for the cause of the ban was that Bing incorrectly thought that I was hosting stolen content, as another domain had reuploaded the book. Unfortunately, the book is licensed under CC BY-SA 4.0, which means I am unable to use copyright to take it down.
June-July: Contacted Bing support
At this point, I sent an email to Bing support explaining the situation (UCM000004402991). After a week, I got the following response:
Thank you for writing to Bing Webmaster Support. I have reviewed your site, and I have now reached out to our Product Review Group for further analysis.
In the meantime, you can go through our Bing Webmaster guidelines to check if your site is violating any of the rules cited in Bing Webmaster Guidelines, especially the section Things to Avoid, and thoroughly check your site for any deliberately or accidentally employed SEO techniques that may have adversely affected your standing in Bing and Bing-powered search results.
Googling this, I’ve found the exact same text sent to someone else, so it’s a copy-paste response.
I once again went through the Bing Webmaster Guidelines, but could not find anything that I was doing wrong. After a month, I chased up the email and got the following:
Our engineering team is still looking into this issue. I will keep you posted on this case. Please allow some time on this request
October: rubenwardy.com unbanned, blog.rubenwardy.com remains banned
Without any update from Bing support, rubenwardy.com started appearing in the results albeit heavily penalised. My blog has since remained banned, with the same error message.
Given this, it seems like the problem may be with the blog rather than my root domain and the reuploaded content issue. I’m not sure what my blog could be doing wrong.
I followed up with Bing support, and they replied saying they have been following up with the engineering team and will keep me posted, but I never received an update.
Ideas for why
Reuploaded content
My original idea was that it was due to another site reuploading my modding book and being considered the original version. But given that the root domain was unbanned and the blog domain wasn’t, this looks less likely.
I have now added canonical tags to most of my pages. Unfortunately, the owner of the other site is malicious so I cannot ask them to fix the problem. The book is licensed under CC BY-SA 4.0, which means I am unable to use copyright to take it down.
Lots of inbound links
I’m quite active online and tend to put my website in my profiles and forum signatures. This has resulted in a lot of inbound links, especially from minetest.net, which may have resulted in Bing considering it link spam.
I developed and host ContentDB, Minetest’s website to find games, mods, and texture packs. I placed a link to rubenwardy.com in ContentDB’s footer as a copyright attribution without realising the SEO implications. As ContentDB has over 14,000 pages, this has resulted in a huge number of inbound links. I have now replaced that footer link with one to an about page that links to my site, this will result in a single quality link rather than thousands of poor-quality links. But these links were to rubenwardy.com, not blog.rubenwardy.com - this doesn’t explain why my blog is still banned.
HTML5 Validation issues
I ran an HTML5 validator on my blog and main website, and fixed a couple of small mistakes.
Weird URLs
Google Search Console alerted me about a large number of Not Found (404) URLs. These URLs are bizarre and look like spam. Another site must be creating these links to me, perhaps they’re hoping my site is badly programmed and will return a soft not found with the links present. After searching, I found some threads which confirmed this. As I correctly return a 404, there’s nothing for me to do here.
Conclusion
My blog (blog.rubenwardy.com) remains banned, and the event has negatively impacted the search placement of my root domain; another site that reuploaded some of my content is appearing as the first result when searching for it. This is especially annoying as it’s the most popular thing on my website, many users go looking for it and instead find an outdated version on a dodgy domain.
It appears that Bing banning small sites may be a common issue, I’ve found other people reporting that their site was similarly effected:
There are potentially over a billion websites on the Internet, Bing needs to index and rank them whilst avoiding abuse. Some websites contain illegal or harmful content, others attempt to manipulate the search results. Given the scale of the problem, it is understandable for Bing to sometimes make mistakes.
In the end, Dave Rupert discovered that Bing was incorrectly classifying his site as a spam blog. A spam blog is a website created purely to link to and promote other websites, often with useless information. Dave was able to fix his problem are blogging about it and raising awareness, so hopefully this post will be able to resolve my problems. I’d appreciate suggestions and help resolving this issue.