993

(skeletor is leading by example by adding that unnecessary apostrophe...)

you are viewing a single comment's thread
view the rest of the comments
[-] Potatos_are_not_friends@lemmy.world 53 points 9 months ago* (last edited 9 months ago)

One of my projects was validation for form submission and emojis melted me. I gave up trying to do it from scratch and trusted a library.

[-] agressivelyPassive@feddit.de 43 points 9 months ago

I'm currently in a project where the client has a custom, but not entirely consistent or known subset of utf-8.

They want us to keep the form content as it is, but remove the "bad" characters. Our current approach is to just forward everything as it is and wait for someone to complain. How TF am I supposed to remove a character without changing the message?

[-] Toes@ani.social 14 points 9 months ago

Yeah I had a backend with poor support for anything that wasn't ASCII. So my solution was turning everything into hex before storing it. I wonder if people are still using it.

[-] dan@upvote.au 8 points 9 months ago* (last edited 9 months ago)

Yeah I had a backend with poor support for anything that wasn't ASCII

PHP is like this. Poor Unicode support, but it treats strings as raw bytes so it usually works well enough. It turns out a programming language can take data from a form, save it to a database, then later load and render it, without having to know what those bytes actually mean, as long as the app or browser knows it's UTF-8, for example through a Content-Type header or meta tag.

The tricky thing is the all the standard string manipulation functions (strlen, substr, etc) don't handle Unicode properly at all and they deal with number of bytes rather than number of characters. You need to use the "multibyte" (Unicode-ready) equivalents like mb_substr, but a lot of PHP developers forget to do this and end up with string truncation code that cuts UTF-8 characters in half (e.g.if it's truncating a long title with Emoji in it, it might cut off the title in the middle of the three bytes that represent the Emoji and only leave 1 or 2 of them)

[-] dan@upvote.au 5 points 9 months ago* (last edited 9 months ago)

You just need to ensure you validate character by character (NOT byte by byte) and allow characters in the Emoji Unicode ranges (which are well-defined in the Unicode standard). Using a library is a great idea though.

[-] GTG3000@programming.dev 4 points 8 months ago

Why not simply discard them?

this post was submitted on 28 Feb 2024
993 points (97.2% liked)

Memes

8393 readers
934 users here now

Post memes here.

A meme is an idea, behavior, or style that spreads by means of imitation from person to person within a culture and often carries symbolic meaning representing a particular phenomenon or theme.

An Internet meme or meme, is a cultural item that is spread via the Internet, often through social media platforms. The name is by the concept of memes proposed by Richard Dawkins in 1972. Internet memes can take various forms, such as images, videos, GIFs, and various other viral sensations.


Laittakaa meemejä tänne.

founded 2 years ago
MODERATORS