MS CS – Machine learning assignment

please see attached document

CS Machine Learning

Don't use plagiarized sources. Get Your Custom Essay on
MS CS – Machine learning assignment
Just from $13/Page
Order Essay

Homework

1

– Theory

Keywords: Boolean functions, mistake bounds, PAC learning

Instructions: Please either typeset your answers (LATEX recommended) or write them very clearly
and legibly and scan them, and upload the PDF on edX. Legibility and clarity are critical for fair
grading.

1. Let D be an arbitrary distribution on the domain {−1, 1}n, and let f, g : {−1, 1}n → {−1, 1}
be two Boolean functions. Prove that

Px∼D[f(x) 6= g(x)] =
1− Ex∼D[f(x)g(x)]

2

.

Would this still be true if the domain were some other domain (such as Rn, where R denotes
the real numbers, with say the Gaussian distribution) instead of {−1, 1}n? If yes, justify your
answer. If not, give a counterexample.

2. Let f be a decision tree with t leaves over the variables x = (x1, . . . , xn) ∈ {−1, 1}n. Explain
how to write f as a multivariate polynomial p(x1, . . . , xn) such that for every input x ∈
{−1, 1}n, f(x) = p(x). (You may interpret −1 as FALSE and 1 as TRUE or the other way
round, at your preference.) (Hint: try to come up with an “indicator polynomial” for every
leaf, i.e. one that evaluates to the leaf ’s value if x is such that that path is taken, and 0
otherwise.)

3. Compute a depth-two decision tree for the training data in table 1 using the Gini function,
C(a) = 2a(1− a) as described in class. What is the overall accuracy on the training data of
the tree?

X Y Z Number of positive examples Number of negative examples

0 0 0 10 20
0 0 1 25 5
0 1 0 35 15
0 1 1 35 5
1 0 0 5 15
1 0 1 30 10
1 1 0 10 10
1 1 1 15 5

Table 1: decision tree training data

4. Suppose the domain X is the real line, R, and the labels lie in Y = {−1, 1}, Let C be the
concept class consisting of simple threshold functions of the form hθ for some θ ∈ R, where
hθ(x) = −1 for all x ≤ θ and hθ(x) = 1 otherwise. Give a simple and efficient PAC learning
algorithm for C that uses only m = O(1� log

1
δ ) training examples to output a classifier with

error at most � with probability at least 1− δ.

1

gchourasia
Cross-Out

5. In this problem we will show that mistake bounded learning is stronger than PAC learning,
which should help crystallize both definitions. Let C be a function class with domain X =
{−1, 1}n and labels Y = {−1, 1}. Assume that C can be learned with mistake bound t using
algorithm A. (You may also assume at each iteration A runs in time polynomial in n, as well
as that A only updates its state when it gets an example wrong.) The concrete goal of this
problem is to show how a learner, given A, can PAC-learn concept class C with respect to
any distribution D on {−1, 1}n. The learner can use A as part of its output hypothesis and
should run in time polynomial in n, 1/�, and 1/δ.

To achieve this concrete goal in steps, we will break down this problem into a few parts. Fix
some distribution D on X, and say the examples are labeled by an unknown c ∈ C. For a
hypothesis (i.e. function) h : X → Y , let err(h) = Px∼D[h(x) 6= c(x)].

(a) Fix a hypothesis h : X → Y . If err(h) > �, what is the probability that h gets k random
examples all correct? How large does k need to be for this probability to be at most δ′?
(The contrapositive view would be: unless the data is highly misleading, which happens
with probability at most δ′, it must be the case that err(h) ≤ �. Make sure this makes
sense.)

(b) As we feed examples to A, how many examples do we need to see before we can be sure
of getting a block of k examples all correct? (This doesn’t mean the hypothesis needs to
be perfect; it just needs to get a block of k all correct. Think about dividing the stream
of examples into blocks of size k, and exploit the mistake bound. How many different
hypotheses could A go through?)

(c) Put everything together and fully describe (with proof) a PAC learner that is able, with
probability of failure at most δ, to output a hypothesis with error at most �. How many
examples does the learner need to use (as a function of �, δ, and t)?

2

What Will You Get?

We provide professional writing services to help you score straight A’s by submitting custom written assignments that mirror your guidelines.

Premium Quality

Get result-oriented writing and never worry about grades anymore. We follow the highest quality standards to make sure that you get perfect assignments.

Experienced Writers

Our writers have experience in dealing with papers of every educational level. You can surely rely on the expertise of our qualified professionals.

On-Time Delivery

Your deadline is our threshold for success and we take it very seriously. We make sure you receive your papers before your predefined time.

24/7 Customer Support

Someone from our customer support team is always here to respond to your questions. So, hit us up if you have got any ambiguity or concern.

Complete Confidentiality

Sit back and relax while we help you out with writing your papers. We have an ultimate policy for keeping your personal and order-related details a secret.

Authentic Sources

We assure you that your document will be thoroughly checked for plagiarism and grammatical errors as we use highly authentic and licit sources.

Moneyback Guarantee

Still reluctant about placing an order? Our 100% Moneyback Guarantee backs you up on rare occasions where you aren’t satisfied with the writing.

Order Tracking

You don’t have to wait for an update for hours; you can track the progress of your order any time you want. We share the status after each step.

image

Areas of Expertise

Although you can leverage our expertise for any writing task, we have a knack for creating flawless papers for the following document types.

Areas of Expertise

Although you can leverage our expertise for any writing task, we have a knack for creating flawless papers for the following document types.

image

Trusted Partner of 9650+ Students for Writing

From brainstorming your paper's outline to perfecting its grammar, we perform every step carefully to make your paper worthy of A grade.

Preferred Writer

Hire your preferred writer anytime. Simply specify if you want your preferred expert to write your paper and we’ll make that happen.

Grammar Check Report

Get an elaborate and authentic grammar check report with your work to have the grammar goodness sealed in your document.

One Page Summary

You can purchase this feature if you want our writers to sum up your paper in the form of a concise and well-articulated summary.

Plagiarism Report

You don’t have to worry about plagiarism anymore. Get a plagiarism report to certify the uniqueness of your work.

Free Features $66FREE

  • Most Qualified Writer $10FREE
  • Plagiarism Scan Report $10FREE
  • Unlimited Revisions $08FREE
  • Paper Formatting $05FREE
  • Cover Page $05FREE
  • Referencing & Bibliography $10FREE
  • Dedicated User Area $08FREE
  • 24/7 Order Tracking $05FREE
  • Periodic Email Alerts $05FREE
image

Our Services

Join us for the best experience while seeking writing assistance in your college life. A good grade is all you need to boost up your academic excellence and we are all about it.

  • On-time Delivery
  • 24/7 Order Tracking
  • Access to Authentic Sources
Academic Writing

We create perfect papers according to the guidelines.

Professional Editing

We seamlessly edit out errors from your papers.

Thorough Proofreading

We thoroughly read your final draft to identify errors.

image

Delegate Your Challenging Writing Tasks to Experienced Professionals

Work with ultimate peace of mind because we ensure that your academic work is our responsibility and your grades are a top concern for us!

Check Out Our Sample Work

Dedication. Quality. Commitment. Punctuality

Categories
All samples
Essay (any type)
Essay (any type)
The Value of a Nursing Degree
Undergrad. (yrs 3-4)
Nursing
2
View this sample

It May Not Be Much, but It’s Honest Work!

Here is what we have achieved so far. These numbers are evidence that we go the extra mile to make your college journey successful.

0+

Happy Clients

0+

Words Written This Week

0+

Ongoing Orders

0%

Customer Satisfaction Rate
image

Process as Fine as Brewed Coffee

We have the most intuitive and minimalistic process so that you can easily place an order. Just follow a few steps to unlock success.

See How We Helped 9000+ Students Achieve Success

image

We Analyze Your Problem and Offer Customized Writing

We understand your guidelines first before delivering any writing service. You can discuss your writing needs and we will have them evaluated by our dedicated team.

  • Clear elicitation of your requirements.
  • Customized writing as per your needs.

We Mirror Your Guidelines to Deliver Quality Services

We write your papers in a standardized way. We complete your work in such a way that it turns out to be a perfect description of your guidelines.

  • Proactive analysis of your writing.
  • Active communication to understand requirements.
image
image

We Handle Your Writing Tasks to Ensure Excellent Grades

We promise you excellent grades and academic excellence that you always longed for. Our writers stay in touch with you via email.

  • Thorough research and analysis for every order.
  • Deliverance of reliable writing service to improve your grades.
Place an Order Start Chat Now
image

Order your essay today and save 30% with the discount code Happy