Thanks for your question about growing olive trees in your area. Depending on where in Jefferson County you live, you are in USDA zones 6a (gets down to -10 deg. F), 6b (gets down to -5 deg. F) or 7a (gets down to 0 deg. F). The type of olive trees that produce olives (Olea Europaea) described in this article, can tolerate the temperatures in zones 8 to 10. Zone 8a gets to an average low temperature of 10 deg. F, so that is 20 degrees higher than the average lowest temperature in your county. And those temperatures are for plants that are in the soil, which tends to insulate the root system better than containers.

So, unless you intend to bring your trees into a room or structure, it will be too cold outside for them to live. As the above source indicates,

"Growing them as houseplants or in containers that must be brought indoors in winter can be difficult. When containers are brought indoors for winter, they should be placed in cool conditions (40-50 degrees F). Trees need at least two months of sub-50 degree F. winter temperatures in order to produce flowers."

These plants are native to the Mediterranean area, with conditions substantially different from Central Oregon.  And, if you're thinking of bringing the pots in during the winter, remember that, under ideal conditions, these trees grow to 30 feet tall.  You're going to need a crane!  But, good luck!  Hope this is helpful information.

We live in Jefferson OR a small town in Marion County Near Albany 
With us being in the Willamette Valley and not central Oregon could you re answer the question 

Thanks for clarifying. Here’s an answer (second in article) that is about Willamatte Valley growing: http://www.oregonlive.com/hg/index.ssf/2016/12/can_i_grow_olive_trees_grow_in.html The same issues apply in the valley as they do east of the mountains: the soil temperature in pots is typically colder than in the ground. And bringing a pot inside will become a chore. But, as Brooke says, you might want to try. Good luck.