Swift Web Scraping: URLSession, Alamofire, SwiftSoup, and Kanna

Swift is not the language you would pick to build a scraping pipeline from scratch — Python and Node have a decade more tooling. But plenty of Swift developers need to pull data off a page anyway: an iOS app reading a site that never shipped an API, a macOS menu-bar tool that watches a price, a Vapor service whose whole stack is already Swift. This guide covers that job honestly: which HTTP client to use, how to decode what comes back, how to parse HTML with SwiftSoup or Kanna, what WKWebView can and can't do for JavaScript-rendered pages, and what App Review will say about the result.

Everything here targets Swift 6 and Alamofire 5.12.

Key Takeaways

• Modern URLSession with async/await covers most scraping needs with zero dependencies — reach for Alamofire when you want its interceptors, retries, and validation, not by reflex
• responseDecodable / serializingDecodable with Codable is the current Alamofire JSON idiom; responseJSON is deprecated and slated for removal in Alamofire 6
• SwiftSoup is a Swift port of Java's jsoup — CSS selectors, pure Swift, works on Linux. Kanna wraps libxml2 and adds XPath
• Redirects are followed automatically by both clients; to inspect or block them use Alamofire's Redirector or URLSessionTaskDelegate
• WKWebView can render JavaScript pages, but it needs a UI process, runs on the main actor, and doesn't exist server-side — it's a per-page tool, not a pipeline
• App Review rejects thin website wrappers under Guideline 4.2, and a scraper that's broken on review day is a rejected build

Does this job belong in Swift at all?

Worth asking before you write any code, because the answer is often no.

Swift is a reasonable choice when:

• The scraping happens inside an app you're already shipping — an iOS client pulling a schedule off a site with no public API, where round-tripping through your own backend isn't worth the operational cost
• You're building a macOS tool or CLI and want a single compiled binary with no runtime to install
• Your backend is Swift already (Vapor, Hummingbird) and adding a Python service is the bigger cost

Swift is the wrong choice when:

• You need a crawler with scheduling, deduplication, and retry semantics — that's Scrapy's job, and nothing in Swift is close
• The target renders everything client-side and you need it at scale — see the WKWebView section below for why
• You need proxy rotation, CAPTCHA handling, and fingerprint management. URLSession's proxy support on Apple platforms is awkward at best (connectionProxyDictionary is unofficial-feeling and inconsistently honored), and none of the anti-blocking ecosystem exists in Swift

A common and sensible middle ground: keep the fetching problem out of the app entirely and call an HTTP API that returns rendered HTML or extracted fields, so the Swift side is just URLSession and Codable. That's the pattern in the last section.

HTTP: URLSession or Alamofire?

Since iOS 15 / macOS 12, URLSession has async/await built in, and that closed most of the gap that made Alamofire feel mandatory:

let url = URL(string: "https://example.com/products")!
var request = URLRequest(url: url)
request.setValue("Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0)", forHTTPHeaderField: "User-Agent")
request.timeoutInterval = 30

let (data, response) = try await URLSession.shared.data(for: request)

guard let http = response as? HTTPURLResponse, (200..<300).contains(http.statusCode) else {
    throw ScrapeError.badStatus
}
let html = String(decoding: data, as: UTF8.self)

That's the whole fetch. No dependency, no build-time cost, and it works identically on Linux through swift-corelibs-foundation.

What Alamofire actually adds, once you look past the nicer syntax:

What Alamofire does not give you: background transfers. Alamofire explicitly does not support background URLSession configurations, so downloads that must survive app suspension have to use URLSession directly.

Rule of thumb: a handful of requests, use URLSession. Retry policy, auth refresh, and request logging across many endpoints, use Alamofire and stop re-implementing them.

Installing Alamofire

Swift Package Manager, in Package.swift:

dependencies: [
    .package(url: "https://github.com/Alamofire/Alamofire.git", from: "5.12.0")
]

Or in Xcode: File → Add Package Dependencies and paste https://github.com/Alamofire/Alamofire.git.

Alamofire 5.10 added full Swift concurrency support with Sendable requirements, and 5.11 raised the floor to Xcode 16 and the Swift 6 compiler. If you're upgrading an older project and see a wall of concurrency warnings, that's the cause — most public APIs carry @preconcurrency for compatibility, but your own captured state has to be Sendable.

import Alamofire

JSON responses with Alamofire

This is what most people arrive for, so it's worth being precise about which API is current.

Define Codable models

struct Product: Codable {
    let id: Int
    let name: String
    let price: Double
    let inStock: Bool
    let releasedAt: Date?

    enum CodingKeys: String, CodingKey {
        case id, name, price
        case inStock = "in_stock"
        case releasedAt = "released_at"
    }
}

responseDecodable (closure-based)

AF.request("https://api.example.com/products/1")
    .validate()
    .responseDecodable(of: Product.self) { response in
        switch response.result {
        case .success(let product):
            print("\(product.name): $\(product.price)")
        case .failure(let error):
            print("Failed: \(error)")
        }
    }

serializingDecodable (async/await — prefer this)

func fetchProduct(id: Int) async throws -> Product {
    try await AF.request("https://api.example.com/products/\(id)")
        .validate()
        .serializingDecodable(Product.self)
        .value
}

.value throws on failure, so the whole thing composes with try await and ordinary Swift error handling. Arrays work the same way — serializingDecodable([Product].self).

Don't use responseJSON

responseJSON hands you Any that you downcast by hand:

// Deprecated since Alamofire 5.5, slated for removal in Alamofire 6.
AF.request(url).responseJSON { response in
    if let json = response.value as? [String: Any] {
        let name = json["name"] as? String   // stringly-typed, silently nil on drift
    }
}

It's deprecated, it loses type safety, and it turns a schema change into a nil rather than an error you can log. If you genuinely need to inspect unknown JSON, decode into a JSONSerialization object or a custom AnyDecodable explicitly — don't reach for a deprecated serializer to get there.

Custom decoders

Snake-case keys and ISO 8601 dates are common enough that a shared decoder beats per-model CodingKeys:

let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
decoder.dateDecodingStrategy = .iso8601

let product = try await AF.request(url)
    .validate()
    .serializingDecodable(Product.self, decoder: decoder)
    .value

Errors that are actually informative

The failure you'll hit most is a DecodingError wrapped in AFError.responseSerializationFailed, usually because the server returned an HTML error page where JSON was expected. Unwrap it rather than printing error.localizedDescription, which flattens the useful part:

do {
    let product = try await AF.request(url).validate()
        .serializingDecodable(Product.self).value
} catch let error as AFError {
    switch error {
    case .responseValidationFailed(reason: .unacceptableStatusCode(let code)):
        print("HTTP \(code)")
    case .responseSerializationFailed(reason: .decodingFailed(let underlying)):
        print("Decoding failed: \(underlying)")   // says which key, and why
    default:
        print("Request failed: \(error)")
    }
}

Sending JSON

struct SearchQuery: Encodable { let term: String; let page: Int }

let results = try await AF.request(
    "https://api.example.com/search",
    method: .post,
    parameters: SearchQuery(term: "swift", page: 1),
    encoder: JSONParameterEncoder.default
).validate().serializingDecodable([Product].self).value

For form posts, swap the encoder for URLEncodedFormParameterEncoder.default — that's the application/x-www-form-urlencoded body most old-school HTML forms expect.

Redirects

Both clients follow redirects automatically, up to a limit, which is usually what you want and occasionally exactly what you don't — login flows in particular, where the Set-Cookie on a 302 is the thing you're after.

Alamofire uses a RedirectHandler; the built-in Redirector covers the three useful policies:

// Follow (the default), don't follow, or rewrite the request mid-flight.
let session = Session(redirectHandler: Redirector(behavior: .doNotFollow))

let response = await session.request("https://example.com/login")
    .serializingData()
    .response

if let location = response.response?.headers["Location"] {
    print("Would have gone to \(location)")
}

.modify gives you the redirected URLRequest before it's sent, so you can strip a header, keep a cookie, or return nil to stop:

let redirector = Redirector(behavior: .modify { task, request, response in
    var request = request
    request.setValue("https://example.com/", forHTTPHeaderField: "Referer")
    return request
})

URLSession does the same through the delegate — note that returning nil for newRequest stops the redirect and hands you the 3xx response:

final class RedirectBlocker: NSObject, URLSessionTaskDelegate {
    func urlSession(_ session: URLSession, task: URLSessionTask,
                    willPerformHTTPRedirection response: HTTPURLResponse,
                    newRequest request: URLRequest) async -> URLRequest? {
        nil
    }
}

let (data, response) = try await URLSession.shared.data(for: request,
                                                        delegate: RedirectBlocker())

Two things that surprise people: URLSession caps redirects at 20 and there's no public knob to change it, and both clients drop the Authorization header on a cross-host redirect. That's a security behavior, not a bug — if you need credentials to survive a hop, set them yourself in the modified request.

Parsing HTML: SwiftSoup

URLSession and Alamofire give you a String of HTML. Regular expressions are not the next step — HTML isn't regular, and a nested tag or an unquoted attribute will break your pattern in a way that's invisible until production. Use a parser.

SwiftSoup is a direct port of Java's jsoup — same API shape, same lenient parsing of real-world broken markup. It's pure Swift with no C dependencies, which means it also builds on Linux.

dependencies: [
    .package(url: "https://github.com/scinfu/SwiftSoup.git", from: "2.6.0")
]

Everything in SwiftSoup throws, because a bad selector is a runtime error:

import SwiftSoup

let doc = try SwiftSoup.parse(html)

// CSS selectors, same syntax you'd use in the browser console
let titles = try doc.select("h2.product-title")
for title in titles.array() {
    print(try title.text())
}

// Single element
if let price = try doc.select("span.price").first() {
    print(try price.text())          // "$29.00" — visible text, entities decoded
    print(try price.html())          // inner HTML
    print(try price.attr("data-cents"))  // "" if absent, never nil
}

// Attributes and traversal
for link in try doc.select("a[href]").array() {
    let href = try link.attr("href")
    let absolute = try link.attr("abs:href")   // resolved against the base URL
    print("\(try link.text()) -> \(absolute)")
}

abs:href only works if you gave the parser a base URL: SwiftSoup.parse(html, "https://example.com/page"). Without it you get the raw relative path, which is a quiet source of broken links downstream.

Three more things worth knowing:

• Selector support is jsoup's, which is broader than CSS: :contains(text), :matches(regex), :has(> img), and :eq(n) all work. Our CSS selectors cheat sheet covers the standard ones.
• text() vs ownText() — text() includes descendants, ownText() doesn't. Scraping a <li>Price: <b>$29</b></li>, text() gives "Price: $29" and ownText() gives "Price:".
• SwiftSoup also sanitizes. try SwiftSoup.clean(userHTML, Whitelist.basic()) strips scripts and unsafe attributes — useful if you're rendering scraped HTML anywhere.

Parsing HTML: Kanna and XPath

Kanna takes the other approach: it wraps libxml2, so it's fast, battle-tested, and — the reason to pick it — supports XPath as well as CSS.

dependencies: [
    .package(url: "https://github.com/tid-kijyun/Kanna.git", from: "6.1.0")
]

import Kanna

let doc = try HTML(html: html, encoding: .utf8)

// CSS
for item in doc.css("div.product") {
    print(item.at_css("h2")?.text ?? "")
}

// XPath — the thing SwiftSoup can't do
for node in doc.xpath("//table[@id='prices']//tr[position() > 1]/td[2]") {
    print(node.text ?? "")
}

// Axes: select by a sibling's content, which CSS has no way to express
let value = doc.at_xpath("//th[text()='SKU']/following-sibling::td[1]")?.text

That last selector is the case for Kanna in one line. Spec tables and definition lists are laid out as "label cell, then value cell," and CSS has no sibling-by-content axis. If your targets are table-heavy, Kanna will save you a lot of index arithmetic. Our XPath cheat sheet has the full axis syntax.

Kanna's API returns optionals rather than throwing (node.text is String?, node["href"] is String?), which reads more naturally in Swift but makes it easier to swallow a selector that silently matched nothing. Assert on empty results in tests.

Pick SwiftSoup by default; add Kanna when you want XPath.

A complete scraper

Putting the pieces together — URLSession for fetching, SwiftSoup for parsing, structured concurrency for a handful of pages at once. No Alamofire here, deliberately, to show how little you need:

import Foundation
import SwiftSoup

struct Article: Sendable {
    let title: String
    let url: String
    let summary: String
}

enum ScrapeError: Error {
    case badStatus(Int)
    case notHTML
}

func fetchHTML(_ url: URL) async throws -> String {
    var request = URLRequest(url: url)
    request.setValue("Mozilla/5.0 (compatible; MyApp/1.0)", forHTTPHeaderField: "User-Agent")
    request.setValue("text/html,application/xhtml+xml", forHTTPHeaderField: "Accept")
    request.timeoutInterval = 20

    let (data, response) = try await URLSession.shared.data(for: request)
    guard let http = response as? HTTPURLResponse else { throw ScrapeError.notHTML }
    guard (200..<300).contains(http.statusCode) else {
        throw ScrapeError.badStatus(http.statusCode)
    }
    return String(decoding: data, as: UTF8.self)
}

func parseArticles(_ html: String, baseURL: String) throws -> [Article] {
    let doc = try SwiftSoup.parse(html, baseURL)
    return try doc.select("article.post").array().compactMap { element in
        guard let link = try element.select("h2 a").first() else { return nil }
        return Article(
            title: try link.text(),
            url: try link.attr("abs:href"),
            summary: try element.select("p.excerpt").first()?.text() ?? ""
        )
    }
}

func scrape(pages: [URL]) async throws -> [Article] {
    try await withThrowingTaskGroup(of: [Article].self) { group in
        for page in pages {
            group.addTask {
                let html = try await fetchHTML(page)
                try await Task.sleep(for: .milliseconds(500))   // be polite
                return try parseArticles(html, baseURL: page.absoluteString)
            }
        }
        return try await group.reduce(into: []) { $0 += $1 }
    }
}

A few notes on the concurrency, because this is where Swift 6 will argue with you. Article is Sendable so it can cross the task boundary; SwiftSoup.Document is not, which is why parsing happens inside the task and only the value types come out. And a task group with no limit will fire every request simultaneously — for more than a dozen URLs, add a semaphore or chunk the input. Ten parallel requests at a site that expected one is how a scraper becomes a rate-limit problem.

Two hardening notes:

• String(decoding:as:) assumes UTF-8. Pages that declare charset=iso-8859-1 will come out with mojibake; read the charset from the Content-Type header or the <meta> tag and use String(data:encoding:) with the right String.Encoding.
• Cookies persist automatically in HTTPCookieStorage.shared, which is convenient for session-based logins and surprising when a stale session breaks a later run. For isolation, build a session with an ephemeral configuration.

JavaScript-rendered pages, and WKWebView's real limits

If the HTML you fetch contains an empty <div id="root">, the content is rendered client-side and no amount of header tweaking will produce it. On Apple platforms, WKWebView is the built-in answer:

@MainActor
final class PageRenderer: NSObject, WKNavigationDelegate {
    private let webView = WKWebView(frame: .zero)
    private var continuation: CheckedContinuation<String, Error>?

    func html(from url: URL) async throws -> String {
        webView.navigationDelegate = self
        webView.load(URLRequest(url: url))
        return try await withCheckedThrowingContinuation { continuation in
            self.continuation = continuation
        }
    }

    func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
        Task {
            let html = try? await webView.evaluateJavaScript(
                "document.documentElement.outerHTML"
            ) as? String
            continuation?.resume(returning: html ?? "")
            continuation = nil
        }
    }
}

That works. Here's what it costs, stated plainly:

• didFinish is not "content ready." It fires when navigation completes, not when the app's data has loaded. Most single-page apps need a polling loop that waits for a selector to appear — and the timeout you pick becomes a source of flaky results.
• It's main-actor-bound and UI-process-backed. Each WKWebView spawns a content process. A few are fine; dozens will exhaust memory, and on iOS a backgrounded app has its web content processes killed.
• Off-screen web views throttle. WebKit deprioritizes rendering for views not in the hierarchy, so a WKWebView you never add to a window may never finish laying out. The usual workaround is a zero-alpha view in the hierarchy, which is exactly as fragile as it sounds.
• It does not exist on Linux. Server-side Swift has no WKWebView, and no Swift-native headless browser. Your options there are shelling out to a headless browser in another runtime, or an HTTP API.
• No proxy rotation, no fingerprint control. WKWebView will happily render a bot-detection page for you.

For one page on demand — a share extension pulling metadata, a macOS tool the user triggers — WKWebView is the right tool. For anything continuous, it isn't.

App Store review considerations

If the scraper ships inside an App Store app, the review guidelines matter as much as the code. Briefly, and honestly:

• Guideline 4.2, Minimum Functionality. An app that is mostly a repackaged website gets rejected. Scraping a site and displaying it needs to add something the site doesn't — offline access, notifications, aggregation across sources, a native interaction the web version lacks.
• Guideline 5.2, Intellectual Property. Displaying another company's content, and especially their branding, without permission is a rejection risk and a legal one. This bites hardest for apps built around a single well-known site.
• The target site's terms of service are a separate question from Apple's rules, and scraping doesn't stop being a contract issue because it happens in an app. Our overview of whether web scraping is legal covers the general shape; it isn't legal advice.
• App Review runs your app on a real device on a real day. If your selectors broke because the site redesigned last week, the reviewer sees an empty screen and rejects the build. This is the practical argument for a server-side layer between the app and the site: you can fix a selector in an afternoon, but an app update takes days.
• App Transport Security requires HTTPS. Scraping an HTTP-only site needs an ATS exception in Info.plist, and exceptions get scrutinized at review. Have a justification.

None of this is a reason not to ship. It's a reason to design so that a site change is a server-side fix, not an emergency App Store submission.

Comparison: how to fetch

Keeping the fetching problem out of your app

The two walls every Swift scraper hits are the same two everywhere: JavaScript rendering and getting blocked. Neither has a good in-app answer — you can't ship a headless browser in an iOS app, and one device is one IP.

WebScraping.AI handles both behind an HTTP API, which keeps the Swift side to URLSession and Codable:

struct ProductFields: Codable {
    let name: String
    let price: String
}

func extract(from target: String) async throws -> ProductFields {
    var components = URLComponents(string: "https://api.webscraping.ai/ai/fields")!
    components.queryItems = [
        .init(name: "api_key", value: apiKey),
        .init(name: "url", value: target),
        .init(name: "fields[name]", value: "Product name"),
        .init(name: "fields[price]", value: "Price with currency symbol")
    ]

    let (data, _) = try await URLSession.shared.data(from: components.url!)
    return try JSONDecoder().decode(ProductFields.self, from: data)
}

/ai/fields returns structured data and skips parsing entirely, which also removes the selector-fragility problem described in the App Review section above. If you'd rather keep parsing in Swift, /html returns the fully rendered page for SwiftSoup, and /selected returns just the fragment a CSS selector matches — a small enough response to decode on a phone.

With Alamofire, the same call is a parameters dictionary:

let fields = try await AF.request(
    "https://api.webscraping.ai/ai/fields",
    parameters: [
        "api_key": apiKey,
        "url": target,
        "fields[name]": "Product name",
        "fields[price]": "Price with currency symbol"
    ]
).validate().serializingDecodable(ProductFields.self).value

It's the division of labor that tends to survive: the network and anti-blocking problem on the server, Swift doing what it's good at — type-safe handling of the results in a native app. Whether that's price monitoring or feeding a RAG knowledge base, the app never has to know a site redesigned.

Frequently asked questions

Should I use Alamofire or URLSession in 2026? Start with URLSession. Since async/await landed it handles requests, headers, timeouts, cookies, and JSON decoding in a few lines with no dependency. Move to Alamofire when you find yourself writing retry logic, a header-injection layer, or per-request validation by hand — those are the pieces it genuinely replaces, and reimplementing them badly is the real cost of avoiding it.

How do I parse HTML in Swift? Use SwiftSoup for CSS selectors (a port of Java's jsoup, pure Swift, works on Linux) or Kanna if you need XPath. Both take an HTML string and give you a queryable document. Don't use regular expressions — nested tags and attribute quoting will break patterns in ways that only show up on the pages you didn't test.

Why is my Alamofire request returning nil or an empty result? Three usual causes. The response isn't JSON — a 403 HTML block page decodes to a DecodingError, so unwrap AFError.responseSerializationFailed and read the underlying error. Your Codable keys don't match the payload — add .convertFromSnakeCase or explicit CodingKeys. Or the page renders its content with JavaScript, in which case the HTML you got genuinely doesn't contain the data.

Can I scrape a JavaScript-heavy site from an iOS app? With WKWebView, one page at a time, in the foreground, with a hand-written wait for the content you need. It works for user-triggered lookups. It doesn't work as a background pipeline: web content processes get killed when the app backgrounds, off-screen web views throttle, and there's no proxy control. For anything continuous, render server-side and let the app fetch a result.

Does Alamofire support async/await? Yes. Use the serializing* family — serializingDecodable(_:), serializingString(), serializingData() — and read .value to get the decoded result or throw, or .response for the full response including headers and status. The closure-based response* methods still work and are not deprecated; the async ones are just less code.

How do I stop Alamofire from following a redirect? Build a Session with Redirector(behavior: .doNotFollow), or attach .redirect(using:) to a single request. Use .modify when you want to inspect or rewrite the redirected request instead of blocking it — that's the hook for keeping a cookie or adding a Referer across the hop. Note that the Authorization header is dropped on cross-host redirects by design.

Can I do web scraping in server-side Swift? Yes for static HTML: URLSession and SwiftSoup both work on Linux, and SwiftSoup has no system dependencies. What's missing is the rendering layer — there's no WKWebView and no Swift-native headless browser — so JavaScript-rendered targets need either a browser in another runtime or a rendering API.