While touring Spain, Englishman Oliver Larker (Denholm Elliott) is rescued from the teasing of local children by friendly cab driver Smiley (Peter Lorre). Hiring Smiley to drive him on a sight-seeing tour, Oliver is captivated by an attractive woman in a white, wide-brimmed hat walking down a side street.After stopping at a shop, the unknown woman proceeds down an alley, apparently unaware of a speeding truck bearing down behind her that is deflected when Smiley's cab accidentally turns in front of it. Annoyed by the excited crowd that gathers to discuss the near accident, Oliver steps into the street where he is approached by Johnny Gin (Liam Redmond), the neighborhood drunk, who insists that he witnessed the event and believes the mysterious woman was targeted by the truck. His curiosity piqued, Oliver visits the shop where the woman stopped and asks the proprietor if he might examine the most recently cashed checks.While Oliver is searching for the women who stopped at the shop, Johnny Gin is questioned by Baron Saradin (Paul Lukas), another witness to the near accident, and their interview is watched by a dark-haired American man (Peter Arne). Oliver tracks down the flirty Miss Jordan (Diana Dors) and an ebullient artist, Miss Leonard (Judith Furse), but he is disappointed that neither physically matches the woman in the white hat. Oliver then finds Sally Kennedy (Beverly Bentley) at her hotel, and, discovering a wide-brimmed, white hat in her closet, is convinced that she is the near victim and offers her his protection. Although grateful, Sally insists that her brother lives nearby and will defend her against any threats.Resuming his tour with Smiley, Oliver proceeds to a scenic vista where he sees Johnny Gin wandering down a narrow street far below. As Oliver watches through his binoculars, three large casks of wine are mysteriously pushed onto the street and roll toward the older man, who desperately tries to outrun them. Panicked, Oliver and Smiley hurry down to the alley but by the time they arrive, Johnny Gin is dead.Concluding that Johnny's murder indicates Sally is in real danger, Oliver orders Smiley to return to her hotel where they see her driving away in a rented red sports car. The pair trail in the cab, unaware that another car, driven by the mysterious American, is following them. After the cab suffers a flat tire and Smiley stops to repair it, Oliver hurries up a nearby hill to follow through his binoculars Sally's progress along the winding roads. Hidden on an opposite hill, Saradin fires rifle shots at Oliver.At the hilltop hotel, Villa Flora, Oliver discovers that the last guest to check in was "Constance Walker." Deducing that Sally has checked in under an assumed name, Oliver and Smiley wait in the bar, unaware that Saradin is already there, as is the American. When a woman (BIllie Miller) answering to "Miss Walker" appears at the bar, Oliver is dismayed that she is not Sally.Upon exploring the hotel grounds, Oliver is confronted by a man who identifies himself as Tommy Kennedy (Leo McKern), Sally's brother. Tommy firmly asserts that his sister is on a cruise and not expected to arrive until the next day. Certain that Sally is on the property, Oliver continues searching and soon finds the red convertible covered by a tarp. After spotting a blonde in a nearby cottage, Oliver makes a dangerous climb to the cottage's balcony which hangs over a cliff. Inside, Oliver is attacked and after a brief skirmish knocks out his assailant, the American. The man revives and tells Oliver and the anxious Sally that he is Richard Fleming, a lawyer and executor of a trust for Sally which designates she will be given three million dollars at the end of the next day, her 25th birthday. Fleming adds that if Sally should die before midnight of the following day, Tommy will receive the inheritance. Excited to have found a motive for the threat against Sally, Oliver maintains that she must be escorted to a safe place. Fleming agrees and Oliver and Smiley drive all night with Sally to Seville, where they hope to lose themselves in the crowds gathered for a fiesta.That afternoon, the trio is shot at while walking and, declaring that she cannot put them at further risk, Sally dashes away. Oliver goes after her and the couple is nearly struck by a rock slide started by Saradin, who has pursued them. Returning to the cab, the trio sets off into the countryside, but when Sally dabs on a bit of perfume, Oliver grows suspicious. When the cab stalls, Oliver accuses Sally of being an impostor as the perfume does not match the "Scent-of-Mystery" perfume he learned Sally Kennedy prefers. Startled, Sally pulls a gun, orders the men from the car and steals the cab.On a nearby hillside, Saradin sees the cab and shoots out a tire, causing the cab to plunge over the mountainside with the Sally impostor still inside to her death. Saradin then chases Oliver and Smiley, firing at them as they hurry through a cave and over steep, narrow passages. Darting into a train tunnel, Oliver and Smiley just avoid a speeding train which kills Saradin.Wondering if Fleming is aware that Sally was a fraud, Oliver determines to find the lawyer, and he and Smiley walk into a local village where they meet an Englishman. Upon learning that Oliver is "Lucky" Larkin, a famed WW II flyer, the man happily loans him his airplane, which Oliver flies back toward the Villa Flora. On the flight, Oliver spots Tommy's van headed toward another town and trails him, landing the airplane near an ancient fortress, Tommy's destination.Oliver notices with a start that a woman in a white hat is waiting in one of the fortress towers and fears Tommy is about to murder his sister. Seeing Fleming's car pull up nearby, Oliver and Smiley are relieved, but when Smiley fills and lights his pipe, the scent causes Oliver to race after the lawyer and knock him out. Oliver then explains to the startled Tommy that Smiley's distinctive American tobacco blend, given him by Fleming at the hotel, matched a scent of smoke he smelled at the truck accident scene. The men then notice a tow truck pulling Smiley's battered cab into the village and Tommy identifies the bruised "Sally" as Fleming's long-time girl friend. Sadly speculating that his wife Margharita, who knew about Sally's trust and was a friend of Saradin's, was likely the motivator behind the plot to get Sally's inheritance, Tommy then takes Oliver to the tower to meet his grateful sister (Elizabeth Taylor) for the first time.