18
submitted 2 weeks ago* (last edited 2 weeks ago) by crashdoom@pawb.social to c/pawbsocial_announcements@pawb.social

Firstly, the maintenance for Pawb.Social services has been completed and we've upgraded Lemmy to 0.19.6-beta.13, and our Mastodon instances to v4.3.0.

Thanks for your patience.


and now, onto the "Toastify is awesome" (because it is. Totally.) postmortem:

Short summary: Crash did a stupid with the config and emails weren’t getting sent (oops). Emails should now be working again, and registration shouldn't require admin intervention.

Longer story:

Lemmy has several configuration options that need to be updated to correspond to the specific instance that's running it. For example, here's a snippet of Pawb.Social's config:

{
  database: {
    host: "the.best.db.url.internal"
    user: "pawbsocial_lemmy"
    database: "pawbsocial_lemmy"
    password: "pawbsocial_do_you_really_think_i_would_leak_this"
    pool_size: 128
  }
  hostname: "pawb.social"
  pictrs: {
    url: "http://pawb-social-pictrs:8080/"
    api_key: "sekrit"
  }
  email: {
    smtp_server: "email-smtp.us-east-2.amazonaws.com:587"
    smtp_login: "pls_send_emails"
    smtp_password: "jeff_im_begging_you"
    smtp_from_address: "Pawb.Social Lemmy <{{domain}}>"
    tls_type: "starttls"
  }
}

Can you spot the problem? (No, it's not my awful password, or the pleading I'm doing with AWS to send emails so y'all stop having email problems.)

Originally, Lemmy was deployed through docker-compose which uses variable replacement to change something like {{domain}} to pawb.social automatically. Crash swapped away from using docker-compose due to repeated issues with trying to deploy the Lemmy changes and opted for a more bare Docker configuration.

Can you guess what didn’t get changed? …Yeah, the {{domain}} part was never removed and replaced with pawb.social so Lemmy correctly caught the error and threw an exception. The unfortunate part is that the thrown exception caused the whole comment process to close the connection to the web proxy server which returned a 502 Bad Gateway, which gave the UI HTML instead of a JSON response. The response handler in the UI then tried to figure out what was wrong, failed to translate the HTML to JSON, and gave the default error message “Toastify is awesome!” (which it is. Objectively.)

HOPEFULLY. This does fix the issue. I can’t guarantee it, but I’ve been able to replicate it by replying to someone else’s comment (which usually triggers an email notification) and after making the change, it successfully sent off the comment with a happy green confirmation toast.

you are viewing a single comment's thread
view the rest of the comments
[-] l_b_i@pawb.social 4 points 2 weeks ago

deleted by creator

Thank you. I though better than to leave this comment as just the first line.

this post was submitted on 29 Oct 2024
18 points (100.0% liked)

Pawb.Social Announcements

432 readers
2 users here now

Official announcements from the admin team of Pawb.Social

founded 1 year ago
MODERATORS