The `unittests` depend on the libraries being tested. They’re linked directly in. It is nonsensical to attempt to build `unittests` standalone.
Rather than adding your contract tests to `unittests`, create a `unittests` subdirectory in your contract directory. Your contract has no need for (and no direct access to) the chainbase library or many of the other parts tested in the top level `unittests`. Imitate the contents of `eosio.token_tests.cpp` for your own test, including making your own copy of the `eosio_token_test` class, renamed to something suitable for your contract. You will need nothing more than what’s in `unittests/CMakeLists.txt` for your own directory’s `CMakeLists.txt`, and you should be able to remove things such as some of the `target_link_libraries()`.