Approx 30 mins ago, I suddenly lost access to lemmy.ca due to DNS resolution failures. I’ve managed to restore access by tossing the known good IP into my computers hosts file, in order to make this post.

It’s worth noting I’m running my own DNS resolver (via PfSense router/firewall software) instead of the typical DNS forwarder offered by my ISP. As a result, my DNS ecosystem is likely a bit more fussy about the “correctness” of the DNS configuration.

My DNS server logs some entries complaining about DS and DNSKEY

debug: Failed to match any usable DS to a DNSKEY.
info: Could not establish a chain of trust to keys for lemmy.ca. DNSKEY IN

An attempt to verify the domain name using delv results in the following:

delv lemmy.ca 
;; broken trust chain resolving 'lemmy.ca/A/IN': 127.0.0.53#53
;; resolution failed: broken trust chain

Running an external DNSSEC check tool, currenly reports DNSSEC related issues. The tool I ran was:

https://dnssec-analyzer.verisignlabs.com/lemmy.ca

This gave me two issues:

None of the 2 DNSKEY records could be validated by any of the 2 DS records

The DNSKEY RRset was not signed by any trusted keys

Did something related to DNS get deployed in the last little while?

In any case, I’ve got a workaround in place and am not stuck, however a site admin may want to be aware of this information.

Thanks!!

  • TruckBC@lemmy.caM
    link
    fedilink
    English
    arrow-up
    8
    ·
    1 year ago

    We’re aware and it should be resolved at this time.

    Did something related to DNS get deployed in the last little while?

    Yes, putting the instance behind cloudflare to protect from DDoS.

    Please let me know if you’re still having issues without your workaround.

    • Farthom@lemmy.caOP
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 year ago

      Thanks for the prompt reply!!

      Things didn’t immediately work, but if you had genuinely changed something I know DNS can take some time for things to trickle down as caches/TTLs expire etc etc etc.

      I’m happy to report things appear to be working normally this morning. I’m glad this was just a transition hiccup and not an outright misconfiguration!

      Thanks so much for all your hard work keeping this thing running for us!

  • nimnim@lemmy.ca
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 year ago

    I still can’t connect to lemmy.ca using my home router unless I use a VPN which is annoying. I thought the server was down the whole day.

    Edit: It had something to do with the AdGuard DNS settings. After changing and then reverting them back, everything now works perfectly fine.

    • Shadow@lemmy.caM
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      I think some DNS servers are holding onto our cached dnssec records for a little longer than they’re supposed to. Every DNS health check I can find is reporting everything is healthy.

      Please let us know if the issue persists!