Well let's see here.
Who was Silverstein speaking to when he used the term "pull it"?
A firefighter.
What context did he make that statement in?
The firefighter had just told him that he was not sure that further firefighting efforts would save the building.
Silverstein, in response to that prognosis, and in conjection with his sentiment that there had already been enough loss of life was obviously telling the firefighter to pull his men out and abandon efforts to save the building.
You have to maintain some common sense about these things.