Do I need to learn the entire C++ Standard Library?

A short answer to this question is no, you do not have to learn the entire C++ Standard Library.

Just because some feature is present in the C++ Standard Library does not necessarily mean we must use or know about it.

What should I learn then?

You should learn about the notable containers and the algorithms operating on those containers. You should also learn about iterators. Some of the notable containers are:

std::vector
std::array
std::set
std::map
std::list
// other containers

Some of the notable algorithms/functions are:

std::find
std::find_if
std::sort
std::count
std::count_if
std::reverse
// other algorithms

Most of these algorithms accept predicates, so you should also learn about lambda expressions which are often used as predicates.

Here is how you could think about it:

"My program needs to have some functionality. Let me explore the C++ Standard Library further and see if it has already been implemented there."

For example, you do not have to create your own sorting algorithm. It has already been implemented in the Standard Library and is called std::sort. You do not have to implement the search function, it has already been implemented in the Standard Library and is called std::find, etc.

For example, you might want to explore the <algorithm> header file further, as it defines many useful functions. But you certainly do not have to know about every function defined inside the <algorithm> header file.

In summary, we do not have to explore the entire C++ Standard Library. We should learn about the notable C++ Standard Library features and the features we will be actually using in our program.