Theory:

     'It's not funny,' she said. 'Even Mommy says he should been a brigadier.'

     It was on the tip of his tongue to make a witty remark (‘Perhaps that’s why he’s still a Colonel’), but he did not want to give offence. They stood on either side of the wall, appraising each other.

     "Do you want to buy it?"

     "No", he said.
 
     'Well' she said,'If you don’t want to buy the house, what are you looking at?’

     ‘I used to live here twenty-five years ago. As a boy. As a young man.... And then my grandmother died, and we sold the house and went away.’
Explanation:
 
The girl was telling it seriously. She felt it was not funny and that even her mother felt that her dad should have been a brigadier by this time.
 
He just wanted to say something but stopped himself from saying as he didn't want anyone to be offended. He intended to make fun of the girl's father as he was still a Colonel. The man and the girl were on either side of the wall assessing each other.
 
The girl asked again if the man wanted to buy the house.
 
He said no, he doesn't want to buy the house.
 
The girl insisted that if he doesn't want to buy the house, then why was he looking at the place.
 
He told her that he used to stay in that house some twenty-five years ago when he was a boy and grew up as a young man there. Later, when his grandmother died, they sold the house and went away from there.
 
Meanings of difficult words:
 
Number
Word
Meanings
1.
 Appraising  Judge.
2.
 Witty  Funny.
3.
 Colonel A rank of officer in the army  above a lieutenant colonel and below a brigadier.
4.
 Brigadier A rank of officer in army, above colonel and below major general.
Reference:
State Council of Educational Research and Training (2018). Term-1 English Standard-9. I Can’t Climb Trees Anymore - Ruskin Bond (Slightly adapted) (pp. 157-160). Published by the Tamil Nadu Textbook and Educational Services Corporation.