Thu 31 January 2019
Some time ago, I read an interesting article from The Guardian about their work with a concept they call "Live Notifications". The general idea is being able to do more with push notifications, turning them into a rich experience with dynamically generated or updated assets. I experimented with this on my own when I wanted a simple way of charting some personal data; I have a server that periodically checks a source, and notifies based on updated findings (yes, this is as generic a description as it can get - it's personal). Rather than generating and storing images on a server that'd only be needed once, couldn't I just dynamically generate them client side?
Turns out, it's not super complicated in the grand scheme of things. Using an iOS Notification Content Extension or iOS Notification Service Extension, it's possible to have a lightweight process running that can act dynamically on received push notifications. This is the key - we'll send a lightweight payload via Apple's Push Notification Service (APNS), and then build and attach an image to the notification before it displays.
Limitations
There are, surprisingly, not too many limitations... but there's one or two to know about.
- Usage of portions of UIKit is pretty much impossible - for instance, UIApplication is out, but you can use CoreGraphics et al as necessary.
- The memory limitations are much smaller and the system is more aggressive in killing your extension if you're not careful, so it's best to keep this efficient. I'd highly recommend sending a default notification with usable title and text, and then customize it as necessary when you do the image.
- If you want to access NSUserDefaults, you'll need to ensure you're using an App Group to communicate between processes properly, as the extension lives separately from your app.
- Oh, and if you use Realm, it's a little tricky to read data in extensions (as of writing this, I don't believe it works properly). I've only used this in situations with NSUserDefaults, Core Data, or SQLite. I'm sure there's a method for Realm, but you're own your own for that.
Building the Extension
For this example, we'll assume you have an iOS app that's properly configured for push notifications. If you're unsure of how to do this, there's enough guides around the internet to walk you through this, so run through one of those first. The example below also makes use of the excellent Charts library by Daniel Gindi, so grab that if you need it.
We'll start with a standard iOS Service Extension, and wire it up to attempt producing an image in the didReceive(...) method. We'll implement three methods, and support throwing up the chain to make things easier - it's less ideal if an extension crashes, because getting it restarted is... unlikely. We'll simply recover "gracefully" from any error, but due to this it's also worth getting right in testing.
import UIKit
import UserNotifications
import Charts
class NotificationService: UNNotificationServiceExtension {
var contentHandler: ((UNNotificationContent) -> Void)?
var bestAttemptContent: UNMutableNotificationContent?
override func didReceive(_ request: UNNotificationRequest, withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void) {
self.contentHandler = contentHandler
bestAttemptContent = (request.content.mutableCopy() as? UNMutableNotificationContent)
if let bestAttemptContent = bestAttemptContent {
bestAttemptContent.title = "\(bestAttemptContent.title) [modified]"
do {
buildChartAttachment(request)
} catch {
// Assuming you sent a "good enough" notification by default, this should be
// safe. We can log here to see what's wrong, though...
print("Unexpected error building attachment! \(error).")
}
contentHandler(bestAttemptContent)
}
}
override func serviceExtensionTimeWillExpire() {
if let contentHandler = contentHandler, let bestAttemptContent = bestAttemptContent {
contentHandler(bestAttemptContent)
}
}
// The three main methods we'll implement in a moment
func renderChartImage() -> UIImage? {}
func storeChartImage(_ image: UIImage?) throws -> URL {}
func buildChartAttachment(_ request: UNNotificationRequest) throws {}
}
Rendering the Chart
For the sake of example, we'll make a very basic LineChart using bogus data. In a real world scenario, you'd want your data to fit into the space of a push notification (2kb - 4kb, which is actually a good amount of space). You could also use a different type of chart, if you wanted. The use cases here are pretty cool - imagine if RobinHood allowed you to, say, see a chart at a glance of how your portfolio is doing. Depending on the performance, that chart could change color or appearance to convey more information at a glance.
Granted, you might not want that much information being on a push notification. Maybe you have prying eyes around you, or something - privacy is probably good to consider if you're reading this and looking to implement it as a feature. The chart below has some settings pre-tuned for a "nice enough" display, but you can tinker with it to your liking.
func renderChartImage() -> UIImage? {
let chartView = LineChartView(frame: CGRect(x: 0, y: 0, width: 320, height: 320))
chartView.minOffset = 0
chartView.chartDescription?.enabled = false
chartView.rightAxis.enabled = false
chartView.leftAxes.enabled = false
chartView.xAxis.drawLabelsEnabled = false
chartView.xAxis.drawAxisLineEnabled = false
chartView.xAxis.drawGridLinesEnabled = false
chartView.legend.enabled = false
chartView.drawGridBackgroundEnabled = true
chartView.drawBordersEnabled = false
chartView.setScaleEnabled(false)
chartView.contentScaleFactor = 2
chartView.backgroundColor = UIColor.black
chartView.gridBackgroundColor = UIColor.green
let dataSet = LineChartDataSet(values: [
ChartDataEntry(x: 1, y: 2),
ChartDataEntry(x: 2, y: 5),
ChartDataEntry(x: 3, y: 7),
ChartDataEntry(x: 4, y: 12),
ChartDataEntry(x: 5, y: 18),
ChartDataEntry(x: 6, y: 7),
ChartDataEntry(x: 7, y: 1)
], label: "")
dataSet.lineWidth = 4
dataSet.drawCirclesEnabled = false
dataSet.drawFilledEnabled = true
dataSet.setColor(UIColor.green)
dataSet.fillColor = UIColor.green
let data = LineChartData(dataSets: [dataSet])
data.setDrawValues(false)
chartView.data = data
return chartView.getChartImage(transparent: false)
}
Note that the size of the chart is hard-coded, and that the scale is manually set. Both are critical for pixel-perfect rendering; the logic could certainly be better (e.g, larger phones really need the scale to be 3), but the general idea is showcased here.
Storing the Image
We now need to attach the image to the notification. We do this using a UNNotificationAttachment, which... requires a URL. Thus, we'll be writing this to the filesystem temporarily. This method attempts to create a temporary directory and write the PNG data from the chart image returned in our prior method.
func storeChartImage(_ image: UIImage?) throws -> URL {
let directory = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true)
try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true, attributes: nil)
let url = directory.appendingPathComponent("tmp.png")
try image?.pngData()?.write(to: url, options: .atomic)
return url
}
Note that, in my testing, simply writing to the same URL over and over again didn't impact multiple notifications - i.e, you're not overwriting an old image that might be on the screen. I've no idea if this will change in later iOS revisions, though, so keep it in the back of your mind!
Putting it all together
With the image saved and ready, we can attach it to the notification and let the system display it to the user.
func buildChartAttachment(_ request: UNNotificationRequest) throws {
let chartImage = renderChartImage()
let url = try storeChartImage(chartImage)
let attachment = try UNNotificationAttachment(identifier: "", url: url, options: nil)
bestAttemptContent?.attachments = [attachment]
}
And voila, you now have a dynamically generated chart. No need to worry about rendering images server side, storing and caching them, or anything like that!
Your chart hopefully looks better than this demo image I found laying around from my test runs. :)
...surely there must be a catch...
Yeah, there's a few things to consider here.
- You're technically pushing the processing requirements to the user's device, but in my testing, this didn't cause significant battery drain over time. If you opt to do this, consider the time interval that you're pushing notifications on.
- As mentioned before, you should design notifications such that they're sent "good enough", in case a notification extension crashes or is killed by the OS for whatever reason. This means ensuring a good default title, body, and so on are sent.
- If you use this for financial data, which would not surprise me as the chief interest here, you should consider making this feature "opt-in" rather than "opt-out". Charts can convey a lot more than text at a glance, and people might not want their information being blown out like that.
But with that all said, it's a pretty cool trick! Due credit goes to The Guardian for inspiring me to look into this. If you find issues with the code samples above, feel free to ping me over email or Twitter and let me know!
Wed 02 January 2019
Recently, I had a friend ask me to glance at some data science work he was doing. He was puzzled why his output, upon attempting to send it to a remote server for processing, was crashing the entire thing. The project was using a pretty standard toolset - Pandas, Numpy, and so on. After looking at it for a minute, I realized he was running into a JSON encoding issue regarding certain data types in Pandas and Numpy.
The fix is relatively straightforward, if you know what you're looking for. I didn't see too much concrete info floating around after a cursory search, so I figured I'd throw it here in case some other wayward traveler needs it.
Creating and Using a Custom JSONEncoder
It all comes down to instructing your json.dumps() call to use a custom encoder. If you're familiar with the Django world, you've probably run into this with django.core.serializers.json.DjangoJSONEncoder. We essentially want to coerce Pandas and Numpy-specific types to core Python types, and then JSON generation more or less works. Here's an example of how to do so, with comments to explain what's going on.
from json import JSONEncoder
class CustomJSONEncoder(JSONEncoder):
def default(self, obj_to_encode):
"""Pandas and Numpy have some specific types that we want to ensure
are coerced to Python types, for JSON generation purposes. This attempts
to do so where applicable.
"""
# Pandas dataframes have a to_json() method, so we'll check for that and
# return it if so.
if hasattr(obj_to_encode, 'to_json'):
return obj_to_encode.to_json()
# Numpy objects report themselves oddly in error logs, but this generic
# type mostly captures what we're after.
if isinstance(obj_to_encode, numpy.generic):
return numpy.asscalar(obj_to_encode)
# ndarray -> list, pretty straightforward.
if isinstance(obj_to_encode, numpy.ndarray):
return obj_to_encode.to_list()
# If none of the above apply, we'll default back to the standard JSON encoding
# routines and let it work normally.
return super().default(obj_to_encode)
With that, it's a one-line change to use it as our JSON encoder of choice:
json.dumps({
'my_pandas_type': pandas_value,
'my_numpy_type': numpy_value
}, cls=CustomJSONEncoder)
Wrapping Up
Now, returning and serializing Pandas and Numpy-specific data types should "just work". If you're the Django type, you could optionally subclass DjangoJSONEncoder and apply the same approach with easy serialization of your model instances.
Mon 31 December 2018
It's the last day of 2018, and a few hours before midnight strikes. Looking back, I didn't get to do as much writing on here for the year as I wanted to (something I'm trying to change in 2019 and beyond). I did, however, manage to find time to do a lot of reading this year; mostly due to long flights, but also from a real effort on my part to get back into the habit.
It seemed like a fitting way to close the year out, then, by listing the books I found insightful and worth grabbing a copy of. I'm opting not to link to Amazon for these, but if you prefer to buy books through there, you should be able to find any of them with a quick search. These are also lightly divided by topics, primarily for some structure.
Privacy
I've generally been a private person, but 2018 in general led to me looking for a better understanding of what privacy really means, both on a consumer and product development level. It's not a topic that can be approached with technology alone; the books below helped me to have an even better understanding of the history of privacy (insofar as US law goes, although much of it is applicable in general).
I found this book while stumbling through the Elliot Bay Book Company in Seattle, and bought it after skimming a few pages. Cyrus does an amazing job outlining the most significant court cases that have shaped the current legal view on privacy, while simultaneously showcasing how the government and general public have failed to keep up with the ethical questions and concerns that have come up in the past few decades as technology has exploded. In general, this was a tough book to put down: it easily messed up my sleep schedule, due to a night or two of reading into the morning hours, and I walked away feeling like I had a better understanding of the privacy landscape.
Rating: 10/10
This book was frustrating, is the best way I can put it. I picked it up on sale after noticing that it was listed as a best seller... and then I regretted it for the following few weeks as I labored through it. The author has a writing style that can be explained best as "stating the obvious for 500 pages". If you have a background in technology, I would skip this; if you're wondering just what's possible with technology, it's probably an okay read, if not a bit hyperbolic.
Rating: 4/10
Much of the internet is opt-in, in a sense - there's generally a privacy policy and terms of service, but it's up to you to read the fine print and decide whether it's good or bad. This book that examines how privacy can be regulated and enforced from the legal side, by requiring product makers to respect privacy in how they build. If you work in tech, touch user data, or have a passing interest in privacy, then this is worth a read.
Rating: 7.5/10
Food
Cooking is a hobby of mine. I wouldn't consider myself professional, but it's amazing for stress relief. It's also a creative outlet for when I can't stand looking at a computer anymore.
This book changed my life: it gave me the final secret to scrambled eggs that I didn't know I needed. While it's worth picking up for this fact alone, the author does a great job illustrating the link between salt, taste, and why so much of what you taste out there tastes bad. I started this on an 8 hour flight and was finished before the end, could not put it down.
Rating: 9/10
A strange book on this list, in that it's easy to look at it more as a cookbook than anything else. I received my copy from a dinner party I attended where the author was giving a presentation, and I guarantee you there's more to it: Turkell does a great job going into the history of vinegar, the different forms out there, and the insane amount of uses it serves. Features a ton of recipes (~100), some of which I still haven't gotten to trying. Highly recommended.
Also, Michael, if you're reading this, my dog literally ate your business card. Hope you found everything you were looking for in Tokyo.
Rating: 7/10
Self
These are books I picked up on a whim, with some level of self improvement in mind.
A collection of articles and essays that help define and increase understanding of emotional intelligence. I originally picked this up to broaden my skills regarding interacting with and leading other people, and I think it helped foster a better way of looking at situations that involve other people. A little bit less about the data and numbers, and more about understanding what it takes to effectively manage and work with people. Recommended.
Rating: 7/10
This is another book where the writing style drove me slightly insane; it definitely felt like the same points being repeated for multiple paragraphs straight. With that said, the content was worth slogging through, and if you can put up with the writing, this is a fun read that'll leave you with some exercises for your brain.
Rating: 6.5/10
2019
Most of these titles were, sadly, only read in the last six months. I'm hoping to make a bigger push in 2019, with a wider range of topics to boot. If you have any recommendations for titles similar to the above, feel free to let me know!
Tue 18 December 2018
Edit as of January 8th, 2019: Twitter announced that they were modifying their last changes, ensuring that some images remain as PNGs! You can read the full announcement here, which details when an image will remain a PNG. Thanks to this, you may not need the trick below - I'll keep it up in case anyone finds it interesting, though.
Edit as of December 26th, 2018: Twitter announced recently that come February 11th, 2019, they'll be enforcing stricter conversion logic surrounding uploaded images. Until then, the below still works; a theoretical (and pretty good sounding) approach for post-Feb-11th is outlined here.
A rather short entry, but one that I felt was necessary - a lot of the information floating in search engines on this is just plain wrong in 2018, and I spent longer than I wanted to figuring this out. It helped me to get my twitter avatar a bit nicer looking, though.
Yeah, that happens. If you upload an image for your profile or banner image, Twitter automatically kicks off background jobs to take that image and produce lightweight JP(E)G variants. These tend to look... very bad. There's a trick you can use for posting photos in tweets where, if you make an image have one pixel that's ~90% transparent, Twitter won't force it off of PNG. It doesn't appear to work on profile photos at first glance, though.
Getting around the problem
Before I explain how to fix this, there's a few things to note:
- Your profile photo should be 400 by 400 pixels. Any larger will trigger resizing. Any smaller, you're just doing yourself a disservice.
- Your profile photo should be less than 700kb, as noted in the API documentation. Anything over will trigger resizing.
- Your profile photo should be a truecolor PNG (24bit), with transparency included. Theoretically you can also leave it interlaced but this didn't impact anything in my testing.
Now, the thing that I found is that the 1 pixel transparency trick doesn't work on profile photos, but if you crop it to be a circle with the transparency behind it, this seems to be enough. If I had to hazard a guess, I'd wager that Twitter ignores transparent pixels unless it deems they seriously matter... as in, there's a threshold you have to hit, or something.
Something like this:
Uploading Properly
For some reason, uploading your profile photo on the main site incurs notably more distorted images than if you do it via an API call. I cannot fathom why this is, but it held true for me. If you're not the programming type, old school apps like Tweetbot still use the API call, so changing the photo via Tweetbot should theoretically do it.
If you're the programming type, here's a handy little script to do this for you:
# pip install twython to get this library
from twython import Twython
# Create a new App at https://dev.twitter.com/, check off "sign in via Twitter", and get your tokens there
twitter = Twython('Consumer API Key', 'Consumer API Secret', 'Access Token', 'Access Token Secret')
# Read and upload the image, see image guidelines in post above~
image = open('path/to/image.png', 'rb')
twitter.update_profile_image(image=image)
Keep in mind that Twitter will still make various sizes out of what you upload, so even this trick doesn't save you from their system - just helps make certain images a tiny bit more crisp. Hope it helps!
Looking for More?
I've been writing for quite some time! You may want to check out the
Archives section for a full list.
