I was reading this interesting paper on importance weighted active learning: http://cseweb.ucsd.edu/~dasgupta/papers/iwal-icml.pdf
I'm having trouble understanding the selection of the rejection threshold. I'm trying to implement a basic example of active learning using sklearn.
As per page 44 of this document: http://hunch.net/~active_learning/active_learning_icml09.pdf I'm looking at how to implement the "implicit" version. However, I don't understand the meaning of "If learn(S ∪ (xt, 1)) and learn(S ∪ (xt, 0)) both return an answer".
Is there anyone who can shed some light or point me to resources that'll help me implement importance weighted active learning?