/dev/trouble
Eric Roller's Development Blog

Framing iPhone 14 screenshots

- Posted in iOS by

When trying to place generated screenshots into device frames (automated by fastlane's frameit), I discovered that, since the iPhone 14 devices have small bezels, the corners of the screenshot can be seen spilling over to outside of the template frames for these devices:

trim_devices = [
    "iPhone 14 Plus",
    "iPhone 14 Pro",
    "iPhone 14 Pro Max",
]

I solved this by cutting the corners off of the screenshot images before running frameit. This can be done by three ImageMagick commands, creating a rounded-rect image with 50-pixel-radius curved corners:

`convert '#{file}' 
    -format 'roundrectangle 1,1 %[fx:w+4],%[fx:h+4] 50,50' 
    info: > mask.mvg`
`convert '#{file}' 
    -alpha transparent 
    -background none 
    -fill white 
    -stroke none 
    -strokewidth 0 
    -draw "@mask.mvg" mask.png`
`convert '#{file}' 
    -alpha set 
    -bordercolor none 
    mask.png 
    -compose DstIn 
    -composite '#{file_name}'`

In my setup, I am operating in a temporary directory (with a language subdirectory, e.g. "en-US"):

Dir.mktmpdir do |tmp|
  path_temp = File.join(tmp, language)
  Dir.mkdir(path_temp)
  puts "Operating in #{path_temp}"

  num_found = 0
  # ...(see below)...
end

Then for each of the devices, depending on whether it is a large iPhone 14 device or not, I either create a corner-cropped image in the temporary directory or place a copy of - or rather a symbolic link to - the original image:

path_shot = File.absolute_path(File.join(
      "..", output_directory, language
  ))

run_devices.each do |device|
  Dir.glob(File.join(path_shot, device + "-*.png")).each do |file|
    file_name = File.basename(file)
    Dir.chdir(path_temp) do
      if trim_devices.include? device
        # Cut off rounded corners using ImageMagick:
        puts "Rounded frame for: #{file_name}"
        `convert '#{file}' -format 'roundrectangle 1,1 %[fx:w+4],%[fx:h+4] 50,50' info: > mask.mvg`
        `convert '#{file}' -alpha transparent -background none -fill white -stroke none -strokewidth 0 -draw "@mask.mvg" mask.png`
        `convert '#{file}' -alpha set -bordercolor none mask.png -compose DstIn -composite '#{file_name}'`
        Dir.glob("mask.*").each { |mask| File.delete(mask) }
      else
        puts "Linking frame for: #{file_name}"
        File.symlink(file, file_name)
      end
    end
    num_found += 1
  end
end

After adding Framefile.json as well as any other needed files into the temporary directory, frameit can be called to do its work there:

frame_screenshots(
    use_platform: "IOS",
    path: tmp
)

Thereafter the "*_framed.png" images need to be copied before the temporary directory is deleted. Here is an example where the result frames are placed in a separate "en-US_framed" directory:

path_fram = File.absolute_path(File.join(
    "..", output_directory, language + "_framed"
))

Dir.glob(File.join(path_temp, "*_framed.png")).each do |file|
  file_name = File.basename(file).sub(/_framed/, "")
  puts "Frame generated: #{file_name}"
  FileUtils.cp(file, File.join(path_fram, file_name))
end

In my setup, fastlane usually runs on a separate computer in my closet. Therefore I find it helpful to add audio messages such that I can hear the status:

sh 'say "Creating screenshots for iOS devices."'

Fastlane's say() command sometimes hasn't worked for me. That's why I have used sh 'say'. However, that was recorded by fastlane as an action. Using this Ruby syntax is not tracked as an action:

`say "Processing images."`

But neither does its produce any text in the log. So I created this Ruby function which prints the message and then speaks it:

def talk(s) puts s; `say "#{s}"` end

With the function anywhere in fastlanes Fastfile, one can just call it like a command:

talk "Framing screenshots"

Speaking Multiple Phrases

This all works nicely, but I discovered that my error message needs to be made more prominent. I might be in the bathroom and don't hear the announcement. That's why I have now changed it to be a lot longer and a lot more annoying.

error do |lane, exception|
  phrases = [
      "Are you serious?",
      "Huston, we've had a problem!",
      "I can't believe it!",
      "I thought you had fixed that.",
      "Is it broken again?",
      "Maybe we can just reboot it?",
      "No! No! No, not again!",
      "Oops! I did it again!",
      "Something smells fishy!",
      "There is a left-over screw here.",
      "There must be a rational explanation for this.",
      "This must be a mistake!",
      "What if we pressed the red button?",
      "What is going on?",
      "What is this switch for?",
      "Where's the beef?",
      "Where is the user manual?",
      "Who pressed the red button?",
      "Who put the dogs out?",
      "You must be joking!",
  ]

  `say "#{ phrases.shuffle.join(" [[slnc 2500]] ") }"`
end

Note that the phrases are shuffled and concatenated by [[sinc 2500]] instructions which add 2.5-second-long pauses between the phrases. I found that in Apple's Speech Synthesis Programming Guide.

Whenever I like to show or hide view elements, I stumble over the hidden() view property:

struct MyView: View {
    var body: some View {
        Text("Visible")

        Text("Not visible")
            .hidden()
    }
}

What perplexes me is why anyone would need this? If it is always hidden, then why add it in the first place??

Well, I guess it could be used to create some well-defined, empty space in the view layout.

Now, this is what I would really have a use for:

struct MyView: View {
    var show_info: Bool

    var body: some View {
        Text("Visible")

        Text("Sometimes visible")
            .hide(!show_info)    // OBS! Doesn't exist!
    }
}

Depending on whether or not the hidden part should be used when laying out the view, a different approach is needed. Either without taking any space:

struct MyView: View {
    var show_info: Bool

    var body: some View {
        Text("Visible")

        if show_info {
            // Not occupying space when hidden
            Text("Sometimes visible")
        }
    }
}

Or occupying space:

struct MyView: View {
    var show_info: Bool

    var body: some View {
        Text("Visible")

        // Taking space, even if transparent
        Text("Sometimes visible")
            .opacity(show_info ? 1.0 : 0.0)
    }
}

Creating screenshots is a tedious task that lends itself to automation. Ideally, it would be preferrable if it were possible to have simple, device-idependent testing scripts. The reality is that many exceptions creep into the code that simulates the UI interactions when generating the screenshots:

if UIDevice.current.userInterfaceIdiom == .phone {
    // Do something different for iPhones.
}

But the real problem is maintaining these scripts. Whether it is iOS SDK changes, new device names, new screen sizes, or indeed new apps features, suddenly the scripts no longer work. Even if everything worked nicely last time, the tiniest of changes will trigger hours of UI scripts re-tuning.

I usually find myself having to re-run fastlane over and over in order to create new screen shots. Quite often, the scripts work well for some devices but fail for others. At that point it would be nice if fastlane could be instructed to only re-run the scripts for the devices that had failed. But fastlane has no option for that.

But it is possible to re-code the Fastfile since it is essentially a Ruby script. All I need is a simple check whether the final screenshot exists for a device. When present, don't re-run the lengthy UI scripts for that device.

So here are the relevant parts of my new ./fastlane/Fastfile; beware custom names like MyApp:

delete_all_screenshots = false

swift_macro_defs =
    "SWIFT_ACTIVE_COMPILATION_CONDITIONS='FASTLANE'"
output_directory = "./screenshots/fastlane"

# All languages and devices to run:
all_languages = [ "en-us" ]

all_devices = [
    "iPad Pro (9.7-inch)",
    "iPad Pro (10.5-inch)",
##  "iPad Pro (11-inch)",
    "iPad Pro (11-inch) (4th generation)",
    "iPad Pro (12.9-inch) (2nd generation)",
    "iPad Pro (12.9-inch) (6th generation)",
    "iPhone 8",
    "iPhone 8 Plus",
    "iPhone 14",
    "iPhone 14 Plus",
    "iPhone 14 Pro",
    "iPhone 14 Pro Max",
    "Apple TV",
    "Apple TV 4K (3rd generation)",
]

platform ::ios do
  desc "Take screenshots (iOS)"
  lane :ios_screenshot do

    all_languages.each do |language|
      # Work out the path to the screenshots.
      # Beware: CWD is the directory of this file!
      path_shot = File.join(
          "..", output_directory, language
      )

      run_devices = Array.new

      all_devices.each do |device|
        # Neither 'Apple TV' nor 'Apple Watch' devices
        next if device.start_with? "Apple"

        unless delete_all_screenshots then
          # If the last screenshot exists, skip this device:
          last_shot = device + "-10_Exercises.lite.png"
          last_shot = device + "-09_Exercises.lite.png" \
              if device.start_with? "iPad"

##        puts "Checking if file exists: #{last_shot}"
          file_shot = File.join(path_shot, last_shot)
          next if File.exists?(file_shot)
        end

        run_devices << device
      end

      next if run_devices.empty?

      snapshot(
          project: "./MyApp.xcodeproj",
          scheme: "MyApp-iOS",
          xcargs: swift_macro_defs,
          devices: run_devices,
          languages: [ language ],
          localize_simulator: true,
          reinstall_app: true,
          headless: false,
          concurrent_simulators: false,
          clean: delete_all_screenshots,
          clear_previous_screenshots:
              delete_all_screenshots,
          stop_after_first_error: true,
          number_of_retries: 0,
          override_status_bar: true,
          test_without_building: false,
          output_directory: output_directory,
          disable_slide_to_type: true,
          skip_open_summary: true,
          only_testing: [
              "UITests-iOD/UITests_Fastlane",
          ],
      )

      delete_all_screenshots = false
    end
  end
end

If you want to recreate all screenshots, you could set delete_all_screenshots to true or, well, delete the files manually.

Also note the SWIFT_ACTIVE_COMPILATION_CONDITIONS compiler flag. This allows me to define a FASTLANE macro that can be used in the app's Swift code:

#if FASTLANE
loadExampleData()
#endif

Time Machine can be configured to exclude folders from your backups (see System Settings > General > Time Machine > Options). I usually exclude these:

  • /Library/Caches
  • ~/Library/Caches
  • ~/Library/Developer/CoreSimulator
  • ~/Library/Developer/XCPGDevices
  • ~/Library/Developer/XCTestDevices
  • ~/Library/Developer/Xcode/DerivedData
  • ~/Library/Developer/Xcode/DocumentationCache
  • ~/Library/Developer/Xcode/iOS DeviceSupport
  • ~/Library/Developer/Xcode/tvOS DeviceSupport
  • ~/Library/Developer/Xcode/watchOS DeviceSupport
  • ~/Library/Logs/CoreSimulator

Also, if you need to make space to be able to install Xcode or a macOS update, you can delete the contents of the above folders (assuming you no longer need old device SDKs).