He may have seen a few going home in the early hours of the morning and heard them telling rude jokes ad stories at the top of their voices. This would have given him the perfect frame for the Miller as most would have been big burley men. The fact he is described as being so ugly may be partly the view that Chaucer has about millers in general and also based on those people that would have been lying drunken in the street.
The Miller has a thombe of gold which makes sense as there would only have been one miller to each village which would mean they could charge what they liked for the flour they made because the people would be highly unlikely and unwilling to go to another village in order to try and get a cheaper price. This therefore makes him seem more realistic and less imagined because Chaucer may have been talking about the one in London and using ugly features of others to make the miller appear ugly. He could have done this as a way of partial revenge to the miller for charging so much.
The pardoner is described in the most grotesque fashion of all three characters. He has heer as yelow as wex. This is a particularly nasty way of describing someone that it makes the reader really able to envisage what it looks like. Therefore Chaucer must have based this on somebody that he had seen because otherwise he would not thought to have used such an unusual simile. Chaucer goes on to say I trowe he were a gelding or a mare. This is possibly one of the greatest insults in the English language and hence supports the view that he based his characters on real people.
Chaucer clearly dislikes the Pardoner otherwise he would not have written about him in such a manner. Chaucer uses irony to describe him because the Pardoner is carrying around fake relics and selling them to poor priests for a months wages. This is against everything that the church and bible stand for. In my opinion Chaucer was using this as a metaphor for his views on the religious system. This is because most of the church at that time had at least a few corrupt members and this was mainly true of the ministers at Rome.
Nearer the end of his description Chaucer is seemingly in awe of the pardoner at how well he is able to tell stories and sing. This is because he finds it so amazing that such a dishonest man can stand in a church and praise god and sell pardons which are essentially sinful in themselves as no one has the power to forgive except for God. This type of person may very well have been met by Chaucer at some point in his life and his utter disgust at the man stayed with him forever. This would undoubtedly influenced his decision as to make the pardoner the most disgusting and unscrupulous members of the pilgrimage.
In conclusion I believe that Chaucer based his characters on real people as the descriptions are in such depth that it is hard to think he could have just made them up. On the other hand as a Fiction writer he has to be very creative and would want his audience to believe the characters as much as possible. I am of the opinion that no one can imagine something completely new that they have never seen before or at least something close to it in nature or appearance.
Therefore even if his attentions were to create complete characters of fiction subconsciously he would have based them on people that he may have seen walking past him in the street. This is unlikely as I feel he would have closely observed different groups and their attitudes very closely in order to get the right appearance and personality that he wanted for his characters.