Ossining Public Library began, in the 1880's, as a collection of 700 books located in the Superintendent's Office at the Park School. In 1892, the School District purchased the entire collection of the private Eagle Circulating Library, which became available when its founders moved out of town. The School now had 2,000 books in its library, and the members of the School Board thought that it was time to form a true public library. They appointed a Board of Library Trustees, and obtained a charter from Albany in February of 1893 as the Sing Sing Public Library.

In 1901 the Library changed its name to the Ossining Public Library. In 1903 the Board of Regents rechartered the library to clarify how trustees are elected to the Board. Beginning at that time, the Library moved its location several times as the book collection grew in size. From the Park School, the Library went to the Twiggar Building on Main Street, and then to the YWCA a few years later. Looking for a permanent home, the Trustees wrote a letter to Andrew Carnegie to ask him for funds to build a new library. He pledged $26,000 for a building if the Trustees could provide the site and commit to an annual budget of at least $2,600 per year. The Trustees purchased the present site on Croton Avenue, known as the George Selleck property, and the library was dedicated in March of 1914. A newspaper at the time boasted that there was not a village of equal size in the State with a finer library building.

The Library was busy from the day it opened. By 1928, the Library had 12,000 books and checked out 63,000 books per year. During the Depression, the Library was even busier, with circulation climbing to 90,000 per year. In 1958, the Library hired its first professional librarian, John A. Burns. In 1959, the cooperative known as the Westchester Library System was established, and Ossining became a charter member. In 1960, the Library hired its first children's librarian, Betsy Rush. Ms. Rush added a new level of service to the library, starting a popular series of programs and story hours for children.

By the early 1960's, the Library was growing faster than ever, and it was starting to stretch the limits of the Carnegie building. There was seating for only 32 patrons, and the shelving was inadequate for the collection of 24,000 books. The Trustees originally intended to build an addition to the present building, but it turned out that a better solution, financially and otherwise, would be to construct an entirely new building. They looked for a new site, but no other property was as centrally located. It was decided to build the new library behind the present one, which would then be demolished. Construction began in December of 1966, and the new building opened in September of 1968. This building had a seating capacity of 75, a 100-seat meeting room, and shelving for over 75,000 books and a variety of audio-visual materials. In the 1990s, the Ossining Public Library joined with other Westchester County libraries in a major undertaking to computerize library operations and to offer Internet access to the public.

The library’s many print and non-print collections continued to grow throughout the 1990s, and the demand for cultural and recreational programming quickly outstripped the capacity of the building’s one meeting room. In 1999, plans were drafted to expand the second floor of the building by 1,500 square feet with the intent of enlarging the children’s room. A referendum was passed to spend over $600,000 on this project, and construction was nearly set to begin when it was discovered that the library’s neighbor to the west, Jackson Lincoln-Mercury, was closing its doors and selling the property. The Junior Room expansion was placed on hold while negotiations commenced to purchase the site, with the intention of pursuing a more substantial expansion. The sale was completed in 2001, at a cost of $700,000, and the architectural firm of Beatty Harvey was selected to design the expansion. The library ultimately decided on a completely new building on the adjacent site rather than adding on to the present building, and plans for the $15.8 million project were presented to the voters in 2003. The measure was overwhelmingly approved, and construction began in early 2005. The former library building was demolished to provide additional parking.

The new 44,000 square foot building opened on March 25, 2007 with a grand opening celebration. About 1,000 people from the community were in attendance. The same year the library charter was amended to change the terms of the library board of trustees from five to three years and the number of trustees from five to seven. In 2011 the charter was again amended to have trustees elected by the voting residents of the school district rather than appointed by the School Board.