Viticci Previews Sky, a New AI-Powered Assistant Coming to macOS… From the Creators of Shortcuts

Ari Weinstein and Conrad Kramer were two of the folks behind Workflow, which Apple bought in 2017 and turned into Shortcuts. The pair left Apple a couple of years ago and launched a startup named Software Applications Incorporated. Today, thanks to an exclusive story over at MacStories, we know what they’re up to. Here’s Federico:

What sets Sky apart from anything I’ve tried or seen on macOS to date is that it uses LLMs to understand which windows are open on your Mac, what’s inside them, and what actions you can perform based on those apps’ contents. It’s a lofty goal and, at a high level, it’s predicated upon two core concepts. First, Sky comes with a collection of built-in “tools”1 for Calendar, Messages, Notes, web browsing, Finder, email, and screenshots, which allow anyone to get started and ask questions that perform actions with those apps. If you want to turn a webpage shown in Safari into an event in your calendar, or perhaps a document in Apple Notes, you can just ask in natural language out of the box.

At the same time, Sky allows power users to make their own tools that combine custom LLM prompts with actions powered by Shortcuts, shell scripts, AppleScript, custom instructions, and, down the road, even MCP. All of these custom tools become native features of Sky that can be invoked and mixed with natural language.

If this sounds too good to be true, well, let me explain and show you how it works.

Sky looks like it will deliver a Mac-centric AI-powered assistant long before Apple gets its act together.

Sky

Linking macOS and LLMs is one thing, but building in support for the Mac’s existing automation tools is next-level, as Ticci writes:

In addition to built-in integrations, you can make your own tools in Sky and use them as native features of the app with natural language. Custom tools can be AppleScripts, shell scripts, and – of course – shortcuts. This is where things go even deeper: custom tools can also be custom prompts for the LLM that tell it to run one or more tools when a specific word or expression is used. Essentially, by virtue of Sky being both an LLM and a desktop app deeply integrated with macOS as opposed to a chatbot living in a web browser, you can make Sky your own by creating any kind of tool you want… with natural language.

What sets Sky apart from traditional chatbots is that it’s the epitome of this concept of hybrid automation I’ve been theorizing about so much lately. In a way, it is reminiscent of the vision behind the agentic Apple Intelligence that was mocked up last year, but it’s available today (well, at least for me) without App Intents. The LLM layer understands what you’re typing and asking; Sky contextualizes it and acts as the glue between the LLM and tools; tools perform actions and pass back results to the LLM. In this virtuous cycle, the result is a personalized, efficient assistant that relies on the smarts provided by an LLM and ties them to the apps you use on your Mac. And it goes even deeper than that: if a tool you want is not available – perhaps because you haven’t created a shortcut for it – Sky can leverage Claude to create its own brand-new tools in the form of shell scripts and AppleScripts.

Apple Adds the iPad to the Self Service Repair Program

Apple PR:

Apple today announced the addition of iPad to Self Service Repair, providing iPad owners with access to repair manuals, genuine Apple parts, Apple Diagnostics troubleshooting sessions, tools, and rental toolkits. Beginning tomorrow, with support for iPad Air (M2 and later), iPad Pro (M4), iPad mini (A17 Pro), and iPad (A16), the launch features components including displays, batteries, cameras, and external charging ports. Today’s announcement joins the expansion of other Apple repair services that further enable customers and independent repair providers to complete out-of-warranty repairs, including new details about the Genuine Parts Distributor program.

I should check on Quinn Nelson.

Why the Original Macintosh Had a Screen Resolution of 512×342, and not 512×384 as One Might Expect

Many classic Macs came with — or supported — displays running at 512×384 pixels, but many compact Macs, ranging from the original 1984 machine up through 1991’s Macintosh Classic II had built-in CRTs running at 512×342 pixels. That covers all black-and-white compact Macs with a 9-inch screen. The later Color Classic and Color Classic II used a 10-inch CRT at a full 512×384.

This came up when I joined John Gruber on The Talk Show. At one point in the show, I rattled off the original Mac’s resolution as being 512×384.

Except… it wasn’t. The original Mac screen ran at 512×342. I remembered the right number and corrected myself a moment later, but given the name of this website, it was pretty embarrassing. This set me off on a journey to understand why Apple made this decision. Why were the displays on early Macs 42 pixels shorter in height than later ones?

After doing a lot of reading, there are several factors to consider when trying to answer this question.

Original Macintosh

Memory

The original Mac had a mere 128 kilobytes of memory. The photo of the original Macintosh in this blog post is 604 KB in size, some 4.7x larger than the entire memory footprint of the 1984 machine. Of course, Apple would improve this with later models, but many design decisions made to accommodate the original Mac would forward for years.

Over at Folklore.org, Andy Hertzfeld wrote a great post walking through several early versions of what would become the Macintosh, including ones with even less memory:

In the beginning of 1982, the original 68000 design was more than a year old, and the software was nowhere near finished, so Burrell [Smith] was afraid some of the trade-offs of the original design were no longer current. He used the expansive canvas of a custom chip, where additional logic was almost free, to update the architecture.

The most important decision was admitting that the software would never fit into 64K of memory and going with a full 16-bit memory bus, requiring 16 RAM chips instead of 8. The extra memory bandwidth allowed him to double the display resolution, going to dimensions of 512 by 342 instead of 384 by 256. He also added bells and whistles like a fancy, DMA-fed sound generator with four independent voices. This was the fourth version of the Macintosh design.

Shipping a computer in the 1980s with a resolution of 384×256 wouldn’t have been too wild. 1982’s Commodore 64 ran at a maximum resolution 320×200. The Apple IIe that shipped in 1983 offered several display modes:

  • 40 or 80 columns text, white-on-black, with 24 lines
  • Low-Resolution: 40×48 (16 colors)
  • High-Resolution: 280×192 (6 colors)
  • Double-Low-Resolution: 80×48 (16 colors)
  • Double-High-Resolution: 560×192 (16 colors)

Computers like the C64 and Apple II had to pull off a lot of tricks to pull off getting graphics on the screen. The Macintosh was going to be powered by a full GUI, and 384×256 would have been just too chunky, so thinking about 128 kilobytes of RAM as an upgrade is a fun twist on the normal take of “Wow, the original Mac was so held back!” Really, it’s amazing that it could do what it did.

That feeling takes on new life when you realize the Mac used a portion of its memory to drive the display. At 512×342, the memory needed to draw the screen was a total of 21.8 KB. Had Apple opted for a 4:3 display running at 512×384, the system would have needed 24 KB for the display. Every byte was precious back in the day. Again, we turn to Andy Hertzfeld:

The original Macintosh only had 128K bytes of RAM (that’s one eighth of a megabyte), so dealing with memory management was usually the hardest part of writing both the system and applications. We allocated around 16K bytes for system use, and another 22K bytes for the 512 by 342 black and white screen, so applications were left with only 90K bytes or so. The bigger ones like MacWrite or MacPaint seemed to be bursting at the seams.

It seems that two things are true:

  • The original Macintosh shipped with more memory than earlier designs
  • Even at 128K, things were extremely tight

Daniel Knight pointed this out when writing about the original Mac:

As Andy Hertzfeld writes, the Mac was only going to have a 256×256 pixel display (a step up from the 280×192 graphics of the Apple II). It wasn’t until January 1981 that the Mac team decided to give the Motorola 68000 a try. A good thing, too, as the first Mac shipped with a 512×342 pixel display, and that would have consumed over 30% of the 64 KB of memory originally envisioned for the low-cost information appliance.

CPU Timings

At the heart of the Macintosh was a Motorola 68000 CPU running at 8 MHz. Just like the 128 kilobytes of RAM, this came with some inherent limitations when paired with the display hardware.

To minimize CRT flicker, Apple worked to achieve a vertical refresh rate of 60 Hz. This meant the CPU spent one-third of its time drawing the display. The original Mac didn’t have a dedicated graphics processor, so the CPU had to spend time loading up that 22K RAM mentioned above, which worked as a display buffer. The machine’s video circuitry would read those 22K and display whatever was present right to the CRT.

Just as with the memory, a taller screen would have taken more resources away from running the Mac’s operating system and programs.

If you are familiar with standard TV formats, you probably have already picked up on the fact that this refresh rate/screen size combination meant the Mac was incompatible with NTSC composite video, which the Apple II supported. (It’s also different than PAL systems.) This let Apple balance performance and picture quality in a way the team saw fit, given the hardware that they had, as Bill Atkinson wrote:

The Apple II displayed white text on a black background. I argued that to do graphics properly we had to switch to a white background like paper. It works fine to invert text when printing, but it would not work for a photo to be printed in negative. The Lisa hardware team complained the screen would flicker too much, and they would need faster refresh with more expensive RAM to prevent smearing when scrolling. Steve listened to all the pros and cons then sided with a white background for the sake of graphics.

Here’s the thing: the original Mac did not run at 8 MHz, but rather 7.83 MHz. This slight tuning meant Apple could time the CPU’s cycles and the CRT’s need for updating more easily.

Square Pixels

Running the 9-inch CRT at 512×342 gave the original Mac a pixels density of 72 PPI, but more importantly, the screen size allowed the Mac to have square pixels.

Apple’s first GUI-powered machine, the Lisa, famously had rectangular pixels, as Andy Hertzfeld covered here:

The Lisa team decided to optimize their display for horizontal resolution, in order to be able to display 80 columns of text in an attractive font. The vertical resolution wasn’t as important, because vertical scrolling works much better for text than horizontal scrolling. The designers decided to endow Lisa with twice as much horizontal resolution as vertical, using a 720 by 360 pixel display, with pixels that were twice as high as they were wide. This was great for text oriented applications like the word processor, but it made things somewhat awkward for the more graphical applications.

When Burrell redesigned the Macintosh in December 1980 to use the same microprocessor as Lisa, the Motorola 68000, it set off shock waves within Apple. Not only was Burrell’s new design much simpler than Lisa, with less than half the chip count, but it also ran almost twice as fast, using an 8 megahertz clock instead of a 5 megahertz clock. Among other advantages was the fact that the Mac’s 384 by 256 pixel display had the identical horizontal and vertical resolution, a feature that we called “square dots”. Square dots made it easier to write graphical applications, since you didn’t have to worry about the resolution disparity.

Hertzfeld goes on to share that the Mac team tried to get the Lisa team to switch to square pixels, bumping the machine’s resolution to a mind-blowing-for-the-time 768×512 pixels, but it wasn’t in the cards, as the Lisa was well on its way to shipping by the time this meeting took place.

Apple Lisa

Of course, the Lisa would end up being a doomed product, and in 1985, Apple rebadged a later revision of the machine as the “Macintosh XL.” It shipped with a software shim called “MacWorks XL” that allowed Mac software to run on the Lisa, but the rectangular pixels made the software appear stretched. To solve this, Apple sold a product named the Macintosh XL Screen Kit, which changed the resolution to 608×432 pixels. This is how the product is described in a document outlining DIY upgrades from Sun Remarketing, a company that was focused on keeping Lisa hardware up and running.

No recently restored Lisa/Mac XL is complete without a Macintosh XL Screen Kit. Unlike the standard 9-inch Macintosh which has square pixels, the stock Lisa/XL has rectangular pixels. With rectangular pixels, circles look like footballs, squares look like spaghetti boxes. The purpose of the Macintosh XL Screen Kit is to square up the pixels. Proportions become exactly the same as on other Macs (1 to 1 ), but the overall display area (608 pixels x 432 pixels) is made roughly the same as a 12-inch Macintosh 11 WYSIWYG monitor (640×480). Standard 9-inch Macs only display 512×342 pixels.

The complete screen modification kit includes new 3A boot ROMs, a new video ROM and a new yoke coil. (Newer software requires System Update 5.0 and MacWorks Plus as well.) Conscientious installation of the complete screen kit requires one to two hours.

Mimicking the Real World

In addition to their square pixels making on-screen graphics look better, the Macintosh team also wanted the computer to be useful for those who needed to print their work. The 72 DPI screen was more than sharp enough for work in applications like MacWrite, MacPaint, and the page layout tools that would follow them. Users could see their work full-sized or at a reduced scale to get a sense of the overall page they were working on. Larger displays would come, but for 1984, 512×342 was plenty.

Everything in Balance

In short, there’s no easy answer to explain why early compact Macs ran at a screen resolution of 512×342. The exact reason for shaving 42 pixels off the vertical resolution is lost to time, but there’s lots of evidence that Apple saw it as advantageous in several different ways.

Apple was doing what it does best: designing a product with the right trade-offs for performance, ease of use, and cost. In a few short years, the Mac would grow to support larger displays, but for 1984, the balance was set correctly.

In the very first edition of Macworld magazine, Matthew Douglas wrote:

Appearances can be deceiving. Most computers display text on one of 24 or 25 “invisible” horizontal lines on the screen. This display is called text mode. To display graphics, the software switches to graphics mode, and the display becomes a field of dots. Each dot, or pixel, is either off (invisible) or on (visible). Of course, a computer may have more than one text mode or two or more graphics modes, or it may be a mixed mode of graphics and text.

The Mac display has only one mode: graphics. The entire screen is made up of dots: 512 dots horizontally and 342 dots vertically, a total of175,104 dots that combine to display everything you’ll ever see on a Mac screen. (Now you know the secret behind the incredible range of type fonts, attributes, and type sizes.)

In the same edition, David Bunnell interviewed Bill Gates, and he was asked about what made the Mac special. Gates — who had previously said the Mac was a computer he would want his mom to try — replied:

The Mac was designed as a graphics machine. Apple didn’t put in a ROM character generator or a bunch of video modes. They put in only one video mode, and that’s the pure bit-mapped, 512-by 342-pixel screen. The monitor was designed into the machine so that they could get extremely crisp pictures and have one integrated system. They knew what the aspect ratio was and how the dots would appear. And they also made sure that the mouse would be used and that the 64K ROM would support very rich graphics interaction.

You can configure a PC with one of the better graphics boards and add a Microsoft mouse and the necessary software, but that’s not the thrust of the machine. The PC is used primarily in its text mode, and to date it’s used mostly without a mouse; you couldn’t get performance or graphics like the Mac’s out of the PC at a comparable price. Although they’re both “turing” machines (that is, they have finite memory), the thrust of the Mac is quite different.

Of all the personal computers available today, the Mac is unique. It’s the first time somebody said, “We don’t need a lot of the things that other personal computers have, so let’s optimize a few areas and make sure the software is designed around them.”

The Talk Show: ‘A Monkey on a Rock’

John Gruber:

Stephen Hackett, proprietor of 512 Pixels and co-founder of Relay (purveyor of many fine podcasts), joins the show. Topics include: IO (or if you will, io), the new joint venture of OpenAI and Jony Ive’s LoveFrom; the sheer fantasy of “Made in America” iPhones; and Fortnite’s return to the US App Store.

Usually May is a pretty quiet time the world of Apple, but this year has proven to be an exception to the rule.

Sponsor: Rogue Amoeba – Strange Name; Great Audio Software

No matter how you use audio on your Mac, Rogue Amoeba has tools for you.

Rogue Amoeba

Start right up top in the menu bar, with SoundSource. It gives you quick controls over any app playing audio on your Mac: control the volume level, redirect to a different output, and even add effects.

Need to record audio? Use their charmingly simple audio recorder Piezo, or its powerful sibling Audio Hijack to turn whatever you can hear into an audio file. Audio Hijack 4.5 even adds new transcription features to turn your recordings into text.

Getting audio from one app into another is easy with Loopback, which creates clever virtual audio devices without the need for bulky audio mixing hardware.

Editing audio files and applying metadata can be done swiftly with Fission, and you can group audio clips together into sets for live playback with Farrago.

Best of all: getting started with Rogue Amoeba’s apps is easier than ever. Each app offers a fully functional free trial, and setup now takes mere seconds, with no restarts or system settings changes required.

Ready to enhance your Mac’s audio capabilities? Head to Rogue Amoeba’s website to download free trials, then use coupon code PIXELS2505 at checkout to save 20% on any purchase. Act fast: this deal expires June 13th, at the end of WWDC.

Dr. Drang Killed a G4 Cube

I love this confession:

One of the Macs my local CompUSA had out on display in its Apple ghetto was a G4 Cube. I had read somewhere that they ran hot, so one day I put my hand a few inches over the Cube to feel the convective flow rising out of the top grille. The machine immediately overheated and shut down to protect itself.

Let me be clear (as clear as the Cube). I didn’t put my hand on the grille—it was at least 3–4 inches above it. But that was enough. And from that day forward, every time I passed that Cube, I put my hand above it and caused a crash.

The Lost Mac Plus Japanese ROM With Built-In Kanji Has Been Found

Pierre Dandumont has saved a long-lost bit of Macintosh history:

If you look for information about the Macintosh Plus and its ROM, you’ll usually find that the ROM has a capacity of 128 KB and that it exists in three revisions. But that’s incorrect: there’s a fourth ROM, 256 KB in size, which includes fonts for kanji (Japanese characters). And I found (and preserved) this ROM.

I had talked about this mysterious ROM a few years ago. It’s documented by Apple in some old documents, but without much detail. According to Apple, the ROM contains fonts for kanji in 12 and 18 points, and they are loaded at startup by KanjiTalk. On a regular Macintosh Plus, you need a floppy disk with the files, which slows down startup and uses some RAM, whereas on a Japanese Macintosh Plus, the font is in ROM and doesn’t take up RAM. You also avoid loading files from a floppy disk, theoretically saving 6 seconds during startup. That’s a conservative estimate, as we’ll see—it assumes you’re not switching disks manually.

Pierre then went on to order not one but two Macintosh Plus logic boards to track down and preserve the ROM. The entire blog post is a delight, as are the photos:

Mac Plus

Riffing on Apple Jam Packs

At Macworld 2004, Apple announced Jam Pack, a collection of additional loops for use in GarageBand:

Jam Pack triples the music content for musicians and aspiring musicians alike with over 2,000 additional loops including many for hip-hop and electronica; over 100 additional software instruments, including a concert grand piano and 12 string guitars; over 100 additional pro-quality effects presets; and 15 additional guitar amps, including surf, grunge, heavy blues and atmospheric.

“GarageBand is like having over 50 musical instruments, a studio of professional musicians and the best recording engineers right at your fingertips,” said Steve Jobs, Apple’s CEO. “And Jam Pack takes GarageBand even further, with even more professional-quality instruments, loops, effects and guitar amps.”

The initial release was compatible with iLife ’04, and was sold for $99. Over time, Apple would add additional products to the lineup, complete with some incredible retail packaging:

Jam Pack Boxes

Expanding the Band

That expansion started in September 2004, the same year the initial Jam Pack shipped:

Apple today introduced two new Jam Packs for its GarageBand music creation software, giving musicians and aspiring musicians alike all of the loops and software instruments they need to create music in the genre of their choice. Jam Pack 2: Remix Tools makes it easy to produce compelling dance, hip-hop and electronica tracks; and Jam Pack 3: Rhythm Section offers songwriters a complete backing band to help build a professional sounding foundation for their rock, alternative and country music.

“GarageBand and the original Jam Pack made it possible for anyone, regardless of their musical background, to easily create great sounding music on their Mac,” said Rob Schoeben, Apple’s vice president of Applications Marketing. “With these two new Jam Packs, musicians and aspiring musicians can delve deeper into their favorite genre. GarageBand just got better.”

Jam Pack 2 came with “more than 2,000 additional loops, 20 new beat kits (Techno, R&B, House, Trance), new synth and sound effect instruments and classic drum machines from Roland (TR-808, TR-909, TR-606, CR-78).” Jam Pack 3 came with over 2,000 more loops and 50 additional instruments:

New drums (Jazz Brushes, Indie Rock, Warehouse Kit), basses (Motown, Liverpool, Progressive Rock, Unplugged) and guitars (Dobro Slide, Bluegrass Banjo, Bluesy Acoustic, Heavy Metal Electric) are a sample of the professional-quality software instruments that come with Rhythm Section.

By early 2005, all four Jam Picks pictured above were available for sale:

Early 2005 Jam Packs website

In 2008, a Voice pack was added to the lineup, as spotted by MacRumors forum member AuthumnSkyline:

Voice Pack

(It’s been a minute since I’ve seen that much Brushed Metal.)

Here’s how Apple described the Voice Pack:

Whether your song needs a soloist, backup singers, or an entire choir, the talent is on
call. Voices gives you over 1,500 Apple Loops featuring professional soloists and choirs in multiple genres and styles. It also includes over 20 software instruments, including voices, choral ensembles, and amazing drum kits built upon the human voice and body.

By this point, the packaging had been redesigned:

2008 Jam Packs

Of course, software in boxes was on its way out within a few years of this redesign, and eventually Apple made these packages available for download for free.

Jam Pack Content Today

These loops live on today, and can be downloaded within Apple’s MainStage and Logic Pro applications via the Sound Library Manager:

Sound Manager

The files are downloaded by the app, and are stored in /Library/Audio/Apple Loops/Apple in separate subfolders for each Jam Pack:

Jam Packs on disk

I love that there’s a little 2004 on my MacBook Pro over two decades later.

In the Cold Light of Day, There Are Questions to Answer About AI-Powered Hardware, Even When Designed by Jony Ive

Ming-Chi Kuo has weighed in on OpenAI’s future hardware. The analyst wrote a long post on X about the hardware he expects to see out of the new partnership between OpenAI and Jony Ive:

My industry research indicates the following regarding the new AI hardware device from Jony Ive’s collaboration with OpenAI:

  1. Mass production is expected to start in 2027.

  2. Assembly and shipping will occur outside China to reduce geopolitical risks, with Vietnam currently the likely assembly location.

  3. The current prototype is slightly larger than the AI Pin, with a form factor as compact and elegant as an iPod Shuffle. The design and specifications may change before mass production.

  4. One of the intended use cases is wearing the device around the neck.

  5. It will have cameras and microphones for environmental detection, with no display functionality.

  6. It is expected to connect to smartphones and PCs, utilizing their computing and display capabilities.

Just as a reminder, this is what that original iPod shuffle looked like when worn:

iPod shuffle

It was exciting to cover the breaking news live on Connected yesterday, but the truth is that Ive and company have a huge challenge ahead of them.

Ive’s previous hits — the iMac, iPod, iPhone, iPad, and Apple Watch — all came to market later than products by other companies, and were all better than existing products. It was easy to see why the iPhone was better than other smartphones, or what made the iPad special. Even though the iPod and Apple Watch were slower out of the gate, they were impressive alternates to what already existed.

Today’s world is far different. No one has shipped an AI product worth buying, and unlike computers and MP3 players and smart phones and tablets and smart watches, the market hasn’t said this is a category that deserves to exist.

To be clear, the presumed failure of the Rabbit R1 and outright failure Humane AI Pin does not mean that there’s no room in the market for an AI-powered device. However, people really like their phones, and creating a product that will compete with the smartphone is a hill no one has successfully climbed to date.

If OpenAI’s future product is meant to work with the iPhone and Android phones, then the company is opening a whole other set of worms, from the integration itself to the fact that most people will still prefer to simply pull their phone out of their pockets for basically any task.

I’m still excited to see what Jony Ive and his team are working on, but just because he’s involved doesn’t mean that the product we will eventually see will be a winner. Joe Rossignol said it well:

It remains to be seen if the device will be a success, or if it will go the way of the AI Pin and other attempts at going beyond the smartphone. OpenAI’s CEO Sam Altman is certainly confident, as he has tested the device at home and believes it will be “the coolest piece of technology that the world will have ever seen.” Quite the claim.