Week 7 muddiestPoint

• Why said CG is not really sensitive to the ranking?

Week 8 reading notes (MIR)

I think this topic user interface and visualization is related to another course I learned “interactive system design”, most accounts of the information access process assume an interaction cycle consisting of query specification, receipt and examination of retrieval results, and then either stopping or reformulating the query and repeating the process until a perfect result set is found. In more details, the standard process can be described according to the following sequence of steps:

Also the berry-pick is illustrated that the information seeking process consisted of a series of interconnected but diverse searches on one problem-based theme. We can found that it is convenient to divide the entire information access process into two main components: search/retrieval, and analysis/synthesis of results. User interfaces should allow both kinds of activity to be tightly interwoven. There are four main types of starting points: lists, overviews, examples, and automated source selection.

Shneiderman identifies five primary human-computer interaction styles. These are: command language, form fill-in, menu selection, direct manipulation, and natural language.

In systems with statistical ranking, a numerical score or percentage is also often shown alongside the title, where the score indicates a computed degree of match or probability of relevance. This kind of information is sometimes referred to as a document surrogate.

User interfaces for information access in general do not do a good job of supporting strategies, or even of sequences of movements from one operation to the next.

Week 6 muddiestPoint

• Why BIR model ignore term frequency and document length lead to not suitable for full text retrieval?

• Recall related to the relevant documents, but why said” Recall is the kitchen sink – you try to get all the relevant documents possible (understanding that you may get many non-relevant documents as well.)”, it should related to the understanding that you may get many non-relevant documents.

Week 7 reading notes (IIR)

Relevance feedback can improve both recall and precision. But, in practice, it has been shown to be most useful for increasing recall in situations where recall is important. This is partly because the technique expands the query, but it is also partly an effect of the use case: when they want high recall, users can be expected to take time to review results and to iterate on the search.

There is some subtlety to evaluating the effectiveness of relevance feedback in a sound and enlightening way.



1. The obvious first strategy is to start with an initial query q0 and to compute a precision-recall graph.

2. A second idea is to use documents in the residual collection (the set of documents minus those assessed relevant) for the second round of evaluation.

3. A third method is to have two collections, one which is used for the initial query and relevance judgments, and the second that is then used for comparative evaluation.

Overall, query expansion is less successful than relevance feedback, though it may be as good as pseudo relevance feedback. It does, however, have the advantage of being much more understandable to the system use

Week 5 muddiestPoint

• How to understand Maximum Likelihood Estimate controbute to the estimation?
• How to set constant to smooth?

Week 6 reading notes (IIR)

    The information need is a little different from the query, the query is more like the SQL query in database. There are some kinds of standard test collections. Such as the Cranfield collection, TREC, GOV2, NTCIR, CLEF, REUTERS, 20 Newsgroups. Precision (P) is the fraction of retrieved documents that are relevant. Recall (R) is the fraction of relevant documents that are retrieved.

    Examining the entire precision-recall curve is very informative, but there is often a desire to boil this information down to a few numbers, or perhaps even a single number. The traditional way of doing this is the 11-point interpolated average precision. In recent years, other measures have become more common. Most standard among the TREC community is Mean Average Precision (MAP), which provides a single-figure measure of quality across recall levels. An ROC curve plots the true positive rate or sensitivity against the false positive rate or (1 − specificity). Here, sensitivity is just another term for recall. The false positive rate is given by f p / ( f p+ t n).

    I also find evaluating the IR system is related to something about interactive systems design. We also need to take consideration about user utility. A/B tests are easy to deploy, easy to understand, and easy to explain to management. Dynamic summaries are generally regarded as greatly improving the usability of IR systems, but they present a complication for IR system design. A dynamic summary cannot be precomputed, but, on the other hand, if a system has only a positional index, then it cannot easily reconstruct the context surrounding search engine hits in order to generate such a dynamic summary. This is one reason for using static summaries.

Week 4 muddiestPoint

• How to understand the difference between document frequency and collection frequency?
• Which skill can come up with a query that produces a manageable number of hits?

Week 5 reading notes (IIR)

    When I look the part of the third classic IR model: the probabilistic model, I find many terms, the obvious order in which to present documents to the user is to rank documents by their estimated probability of relevance with respect to the information need: P(R = 1|d, q). This is the basis of the Probability Ranking Principle (PRP).

    We also have to use some probabilistic term to illustrate and improve the probabilistic model, like Binary Independence Model (BIM), The resulting quantity used for ranking is called the Retrieval Status Value RETRIEVAL STATUS (RSV) maximum likelihood estimate MLE maximum a posteriori (MAP ). Length normalization of the query is unnecessary because retrieval is being done with respect to a single fixed query