DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from JP2022-158387, filed September 30, 2022.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on September 19, 2023 was reviewed and the listed references were noted.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: VEHICLE-INSTALLED IMAGE PROCESSING DEVICE AND IMAGE DISPLAY DEVICE.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “emphasis processing unit configured to generate”, “smoothing unit configured to generate”, “combination unit configured to generate”, “priority calculation unit configured to calculate”, “sharpening unit configured to generate”, “image acquisition unit configured to acquire” and “image processing unit configured to perform” in claims 1-7.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5 and 7 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang et al. (US 9,336,574 B2).
Regarding claim 1, Zhang teaches an image processing device, comprising:
an emphasis processing unit configured to generate an emphasized image yielded by performing an emphasis process on an image captured of surroundings of a vehicle (Zhang, Col. 5, lines 28-42 the term “sharpening filter” refers to a filter utilized to enhance sharpness in the identified region of interest of the image frame, and thus, reduces image blurring. The sharpening filter includes an unsharp masking (USM) filter. Fig. 3 illustrates an exemplary USM filter for sharpness enhancement of a captured raw image. Col. 3, lines 49-59, a raw image is captured by the camera device illustrating the FOV rearward of the vehicle);
a smoothing unit configured to generate a smoothed image yielded by performing a smoothing process on the image (Zhang, Col. 4 line 52 – Col. 5, line 27, One example of spatially-implemented super-resolution includes application of a de-noising filter. As used herein, the term “de-noising filter” refers to a filter to reduce noise within the identified region of interest. A non-limiting exemplary de-noising filter includes a bilateral filter that reduces noise within the identified region of interest through smoothing identified image planes while preserving identified edges within the identified region of interest); and
a combination unit configured to generate, for each of a plurality of regions into which the image is divided, a display image to be displayed in a display using any of the smoothed image, the emphasized image, and the image (Zhang, Col. 6, lines 42-65, The resulting image 309 is perceived as being sharpened by the sharpening filter. In one embodiment, the resulting image 309 can be displayed upon the rearview mirror display device 24 of FIG. 1. In another embodiment, the resulting image 309 in combination with one or more other resulting images of the same scene and having sharpening filters applied thereto, can be subjected to temporally-implemented super-resolution to the resulting images dependent upon a location within each of the resulting images).
Regarding claim 5, Zhang teaches the image processing device according to claim 1, and further teaches wherein the emphasis process is a sharpening process of emphasizing edges of the image (Zhang, Col. 5, lines 28-65 the term “sharpening filter” refers to a filter utilized to enhance sharpness in the identified region of interest of the image frame, and thus, reduces image blurring. The sharpening filter includes an unsharp masking (USM) filter. The intensity threshold controls a minimum intensity change of pixels, e.g., brightness, associated with identified edges that will be sharpened).
Regarding claim 7, Zhang teaches a display device, comprising:
an image acquisition unit configured to acquire an image captured of surroundings of a vehicle (Zhang, Col. 2, lines 33-47, the vision-based imaging system captures images surrounding the vehicle based on the location of one or more vision-based camera devices);
a display unit configured to display a display image based on the image (Zhang, Col. 2 line 48 – Col. 3 line 4, a non-transitory processing device that processes the image data to generate images that can be displayed on a rearview mirror display device); and
an image processing unit configured to perform image processing on the image to generate the display image (Zhang, Col. 2 line 48 – Col. 3 line 4, a non-transitory processing device (e.g., processor) that processes the image data to generate images that can be displayed on a rearview mirror display device),
wherein the image processing unit includes:
an emphasis processing unit configured to generate an emphasized image yielded by performing an emphasis process on the image (Zhang, Col. 5, lines 28-42 the term “sharpening filter” refers to a filter utilized to enhance sharpness in the identified region of interest of the image frame, and thus, reduces image blurring. The sharpening filter includes an unsharp masking (USM) filter. Fig. 3 illustrates an exemplary USM filter for sharpness enhancement of a captured raw image. Col. 3, lines 49-59, a raw image is captured by the camera device illustrating the FOV rearward of the vehicle);
a smoothing unit configured to generate a smoothed image yielded by performing a smoothing process on the image (Zhang, Col. 4 line 52 – Col. 5, line 27, One example of spatially-implemented super-resolution includes application of a de-noising filter. As used herein, the term “de-noising filter” refers to a filter to reduce noise within the identified region of interest. A non-limiting exemplary de-noising filter includes a bilateral filter that reduces noise within the identified region of interest through smoothing identified image planes while preserving identified edges within the identified region of interest); and
a combination unit configured to generate, for each of a plurality of regions into which the image is divided, the display image using any of the smoothed image, the emphasized image, and the image (Zhang, Col. 6, lines 42-65, The resulting image 309 is perceived as being sharpened by the sharpening filter. In one embodiment, the resulting image 309 can be displayed upon the rearview mirror display device 24 of FIG. 1. In another embodiment, the resulting image 309 in combination with one or more other resulting images of the same scene and having sharpening filters applied thereto, can be subjected to temporally-implemented super-resolution to the resulting images dependent upon a location within each of the resulting images).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 9,336,574 B2) in view of Austin et al. (US 2022/0121865 A1, filed October 20, 2020).
Regarding claim 2, Zhang teaches the image processing device according to claim 1, as described above.
Although Zhang teaches applying de-noising and sharpening to an image (Zhang, Col. 4 line 52 – Col. 5, line 42), Zhang does not explicitly teach “a priority calculation unit configured to calculate a degree of priority of the emphasis process or the smoothing process for each of the plurality of partitioned regions into which the image is divided” and “wherein the combination unit selects, for each of pixels of the plurality of partitioned regions, whether to use the smoothed image, the emphasized image, or the image based on the degree of priority, to generate the display image”. However, in an analogous field of endeavor, Austin teaches an aspect is to quickly sharpen image regions/lines in an image in anticipation of a person shifting their head to a new focus vision area. By softening image regions/lines outside of the focus vision area of a driver's vision, the cognitive resources freed up by blurring or softening image regions/lines of task-irrelevant images in the periphery can be reallocated to other brain regions necessary for supporting the driver, to use cognitive resources efficiently while driving a vehicle, especially when performing mission-critical tasks. An abrupt change from softer to sharper image regions/lines may help to draw attention in order to reduce time in focusing on a display image so that the driver can revert back to focusing on the task at hand. Conversely, a gradual, inconspicuous change from sharp to softer image regions/lines may minimize—or potentially eliminate—the drawing of attention to task-irrelevant information in the periphery so that the driver can continue focusing on the task at hand (Austin, Para. [0035]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Zhang with the teachings of Austin by including determining softening or sharpening image regions based on the priority of a driver focusing their vision on that area of the display. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for minimizing or eliminating the drawing of a driver’s attention to task-irrelevant information, as recognized by Austin. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 6, Zhang teaches an image processing device, comprising:
a sharpening unit configured to generate a sharpened image yielded by sharpening edges of an image captured of surroundings of a vehicle (Zhang, Col. 5, lines 28-42 the term “sharpening filter” refers to a filter utilized to enhance sharpness in the identified region of interest of the image frame, and thus, reduces image blurring. The sharpening filter includes an unsharp masking (USM) filter. Fig. 3 illustrates an exemplary USM filter for sharpness enhancement of a captured raw image. Col. 3, lines 49-59, a raw image is captured by the camera device illustrating the FOV rearward of the vehicle).
Although Zhang teaches applying de-noising and sharpening to an image (Zhang, Col. 4 line 52 – Col. 5, line 42), Zhang does not explicitly teach “a priority calculation unit configured to calculate a degree of priority of the sharpening for each of a plurality of partitioned regions into which the image is divided” and “a combination unit configured to select, for each of pixels of the plurality of partitioned regions, whether to use the image or the sharpened image based on the degree of priority, to generate a display image to be displayed in a display”. However, in an analogous field of endeavor, Austin teaches an aspect is to quickly sharpen image regions/lines in an image in anticipation of a person shifting their head to a new focus vision area. By softening image regions/lines outside of the focus vision area of a driver's vision, the cognitive resources freed up by blurring or softening image regions/lines of task-irrelevant images in the periphery can be reallocated to other brain regions necessary for supporting the driver, to use cognitive resources efficiently while driving a vehicle, especially when performing mission-critical tasks. An abrupt change from softer to sharper image regions/lines may help to draw attention in order to reduce time in focusing on a display image so that the driver can revert back to focusing on the task at hand. Conversely, a gradual, inconspicuous change from sharp to softer image regions/lines may minimize—or potentially eliminate—the drawing of attention to task-irrelevant information in the periphery so that the driver can continue focusing on the task at hand (Austin, Para. [0035]).
The proposed combination as well as the motivation for combining the Zhang and Austin references presented in the rejection of Claim 2, apply to Claim 6 and are incorporated herein by reference. Thus, the device recited in Claim 6 is met by Zhang in view of Austin.
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 9,336,574 B2) in view of Austin et al. (US 2022/0121865 A1, filed October 20, 2020), as applied to claims 2 and 6 above, and further in view of Kim et al. (US 2022/0381578 A1, filed February 22, 2022).
Regarding claim 3, Zhang in view of Austin teaches the image processing device according to claim 2, as described above.
Although Zhang in view of Austin teaches performing softening and sharpening based on a driver’s attention to regions of a display (Austin, Para. [0035]), they do not explicitly teach “wherein the priority calculation unit acquires speed information indicating a speed of the vehicle, and calculates the degree of priority based on information of a degree of emphasis of the emphasis process set in advance for each of the plurality of partitioned regions, and the speed information”. However in an analogous field of endeavor, Kim teaches the controller may determine whether a moving speed is less than the reference speed based on the moving speed information, determine whether the distance from the current location (i.e., distance from body of vehicle) to the destination is less than or equal to a set distance based on the current location information and the location information of the destination when it is determined that the moving speed is less than the reference speed, recognized a subject image corresponding to the destination information and control the display device to display an emphasized object image indicating the destination on the image of the recognized subject (Kim, Para. [0164]). Kim teaches the vehicle may further include a speed detector for detecting a travelling speed (i.e., a moving speed of the vehicle) (Kim, Para. [0239]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Zhang in view of Austin with the teachings of Kim by including acquiring speed information indicating the speed of the vehicle and determines if it should display the emphasized object image (i.e., degree of priority) based on the speed and distance from the current location. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for updating a vehicle display based on speed and location, as recognized by Kim. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 4, Zhang in view of Austin further in view of Kim teaches the image processing device according to claim 3, and further teaches wherein the information of the degree of emphasis of the emphasis process is set in advance for each of the plurality of partitioned regions based on a distance from a body of the vehicle (Kim, Para. [0164], determine whether the distance from the current location (i.e., distance from body of vehicle) to the destination is less than or equal to a set distance based on the current location information and the location information of the destination when it is determined that the moving speed is less than the reference speed, recognized a subject image corresponding to the destination information and control the display device to display an emphasized object image indicating the destination on the image of the recognized subject).
The proposed combination as well as the motivation for combining the Zhang, Austin and Kim references presented in the rejection of Claim 3, apply to Claim 4 and are incorporated herein by reference. Thus, the device recited in Claim 4 is met by Zhang in view of Austin further in view of Kim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662