Relative clauses are the clauses which begin with the relative pronouns such as when, where, which, that or who. The noun that precedes, relative pronouns identify or define that noun.
Therefore, to complete the sentence with relative best option would be which was home to many more extinct species.
The Tasmanian tiger was a dog-like creature from the island of Tasmania, which was home to many more extinct species.
In the above sentence, 'which' is the relative pronoun that defines the noun 'The Tasmanian tiger.'