The right-hand side of (5) is a quadratic non linear function of x, whereas
the left-hand side
is a linear function of x. The two sides "do not match," and so the functional
form for the
optimal value function V (•) cannot be linear.
Part c. Here, one simply maximizes the quadratic function of z given by (x-z)z.
solution is z(x) = x/2 and the objective function is (x - x/2)(x/2) = x^2/4.
Part d. The solution to part (c) suggests that V (x) = Kx^2 for some positive
K and that the
optimal policy is z(x) = Cx for some positive C. You can verify that if you set
V (x) = Kx^2
on the right-hand side of the Bellman equation, the right-hand side is still a
of z. The optimal value z(x) will be linear in x, and the optimal objective
function value will
remain quadratic upon substitution. The solution to Part (e) be low will verify
this and provide
an exact solution.
Part e. Consider the right-hand side of the Bellman equation when V (x) = Kx^2.
derivative of the function (x - z)z + dK(x - z)^2 with respect to z and setting
it to zero yields
0 = x - 2z- 2dK(x - z), and thus
Given the value for K, we can express the optimal policy as
The optimal objective function value of the right-hand side of the Bellman
equation is therefore
The right-hand side function must be identical to the left-hand side
function, which is, by
assumption, Kx^2. If our as sumption is correct , then a value of K exists such
Assume d = 1/1.1. We can solve for the value of K and thus C(K) using
approximations. That is, we can define
where C(Ki) is defined in (7). Assuming V0(•) = 0 is identical to setting K0 =
0. This yields
C(0) = 0.5 from (7). Substituting C(0) = 0.5 into the right-hand side of (11)
yields K1 = 0.25.
(We\'re just repeating the myopic solution so far.) The value of C(0.25) is
C(0.25) = 0.3529 into the right-hand side of (11) yields K2 = 0.3235. The value
is 0.2917. Substituting C(0.3235) = 0.2917 into the right-hand side of (11)
yields K3 = 0.3402.
The process continues. The Ki converge to
:= 0.38416876 and C() =
Remark. The identity (11) can be directly solved for K,
but it requires a bit of lengthy algebra.
It turns out that the right-hand side simplifies to the reciprocal of 4(1 - dK).
Thus, the K
value we seek satisfies the identity 1 = 4(1-dK)K or 4dK^2 -4K +1 = 0. The
solution to this
quadratic equation is
Start solving your Algebra Problems
in next 5 minutes!
Download (and optional CD)
Click to Buy Now:
2Checkout.com is an authorized reseller
of goods provided by Sofmath
Attention: We are
currently running a special promotional offer
for Algebra-Answer.com visitors -- if you order
Algebra Helper by midnight of
you will pay only $39.99
instead of our regular price of $74.99 -- this is $35 in
savings ! In order to take advantage of this
offer, you need to order by clicking on one of
the buttons on the left, not through our regular
If you order now you will also receive 30 minute live session from tutor.com for a 1$!
You Will Learn Algebra Better - Guaranteed!
Just take a look how incredibly simple Algebra Helper is:
: Enter your homework problem in an easy WYSIWYG (What you see is what you get) algebra editor:
Step 2 :
Let Algebra Helper solve it:
Step 3 : Ask for an explanation for the steps you don't understand:
Algebra Helper can solve problems in all the following areas:
simplification of algebraic expressions (operations
with polynomials (simplifying, degree, synthetic division...), exponential expressions, fractions and roots
(radicals), absolute values)