Yeah, and if you call it 50 bytes with row headers and padding, that ends up as ~2Mb/update just for notifications.
That sounds like a lot - and it is a fair bit, though the image is probably 2Mb as well. You might even have to multiply it a few times for indexing purposes. But databases are used to dealing with lots of data; and for regular users, an update would be closer to 8Kb. More importantly, it's useful data in that lots of people want to know when Blotch posts an update - which is, like, every three months at this point.
(In fact, most of those listed on Popufur.com tend to post maybe once a day max. Because the most-watched artists tend to do good work, which takes time, their impact on the database may be minimal. Writing 2Mb in a block costs less than writing 256 8kb blocks.)
In theory you could instead do a query over the submissions of the people you're watching and not store any additional data. But doing this all the time costs a lot, too - you end up having to scan back through oodles of large submission records, rather than go directly to the ones you're interested in. This increases your I/O read requirements, and means you need to spend more on RAM and CPU to hold those large datasets - or, be prepared to write them temporarily to disk, which will instantly cost far more than an update by anyone. At that point, looking up the last 36 50-byte entries that are already in an ordered list starts looking a lot more sensible.
Yeah, and if you call it 50 bytes with row headers and padding, that ends up as ~2Mb/update just for notifications.
That sounds like a lot - and it is a fair bit, though the image is probably 2Mb as well. You might even have to multiply it a few times for indexing purposes. But databases are used to dealing with lots of data; and for regular users, an update would be closer to 8Kb. More importantly, it's useful data in that lots of people want to know when Blotch posts an update - which is, like, every three months at this point.
(In fact, most of those listed on Popufur.com tend to post maybe once a day max. Because the most-watched artists tend to do good work, which takes time, their impact on the database may be minimal. Writing 2Mb in a block costs less than writing 256 8kb blocks.)
In theory you could instead do a query over the submissions of the people you're watching and not store any additional data. But doing this all the time costs a lot, too - you end up having to scan back through oodles of large submission records, rather than go directly to the ones you're interested in. This increases your I/O read requirements, and means you need to spend more on RAM and CPU to hold those large datasets - or, be prepared to write them temporarily to disk, which will instantly cost far more than an update by anyone. At that point, looking up the last 36 50-byte entries that are already in an ordered list starts looking a lot more sensible.