DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 2, 3, 10, 11 and 16 are cancelled. Claims 1, 4, 6, 7, 9, 12, 13, 15, 17, 19 and 20 are amended. Claims 1, 4-9 and 12-15 and 17-20 are pending.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 4-7, 9, 12-15 and 17-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Akao (US PG Pub. 20220239876).
Regarding claims 1 and 15, Akao discloses an electronic apparatus comprising:
an image projection unit (projector 2 of fig. 1); and
a processor (control unit 123 of fig. 7 and para. 0065; the projector 2 projects an image of a dot pattern in which circular dots each having a predetermined diameter are regularly arranged in the entire projection range of the projector 2) configured to: control the image projection unit to project a test image (para. 0065; the projector 2 projects an image of a dot pattern in which circular dots each having a predetermined diameter are regularly arranged in the entire projection range of the projector 2) comprising a plurality of markers onto a projection surface (illustrated in fig. 12; and para. 0150; FIG. 12, a small black ellipse that is laterally long represents each dot included in the dot pattern. In addition, a hatched small circle represents a vertex of the region of the projection image.),
identify first information comprising coordinate information of the image projection unit indicating a position of each of the plurality of markers in the test image (para. 0143; Coordinates of the dots on each captured image are detected by the pattern detection processing, and the relationship between the dots on each of the captured images and the dots on the dot pattern is determined), second information based on a captured image capturing the projection surface from an external device (para. 0066; the user uses a camera function of the mobile terminal 1 to capture an image of the dot pattern projected by the projector 2 at a plurality of different image capturing positions. The captured images obtained by the image capturing show, for example, a range including the entire dot pattern projected as the projection image on the projection surface 3) and comprising coordinate information indicating a position of each of the plurality of markers in the captured image (para. 0143; Coordinates of the dots on each captured image are detected by the pattern detection processing, and the relationship between the dots on each of the captured images and the dots on the dot pattern is determined), and
information about a guide area set using a guide graphical user interface (GUI), in the captured image (para. 0075; a guide, the user performs image capturing a plurality of times while changing the image capturing position),
obtain the second information (para. 0115-0116; the plurality of captured images output from the image capturing unit 131 is input to the corresponding point detection unit 151. Each of the captured images output from the image capturing unit 131 shows the dot pattern projected on the projection surface 3 and para. 0117; The corresponding point detection unit 151 stores corresponding point information, which is information indicating the relationship between the corresponding points detected by the corresponding point detection processing, and outputs the corresponding point information to the camera attitude estimation unit 152),
obtain the information about the guide area set in the captured image (para. 0156; The corresponding point information for each of the captured images indicates the relationship between the camera corresponding points of each of the captured images captured at the corresponding position and the projector corresponding points),
obtain third information indicating positions of four vertexes of the test image (illustrated in fig. 12; hatched dots in each corner of the solid dot image) in the captured image based on the first information and the second information (para. 0153; the corresponding point detection unit 151 transforms the projection image shown in the captured image on the basis of the homography matrix so as to obtain a substantially rectangular image having the same shape as that of the input image), and
perform keystone correction (para. 0132; The correction data calculation processing #1 of FIG. 9 is a series of processing for calculating the correction data used for the geometric correction of the input image) so that an input image corresponding to the guide area is projected based on the first information, the second information, and the information about the guide area (para. 0153; the corresponding point detection unit 151 transforms the projection image shown in the captured image on the basis of the homography matrix so as to obtain a substantially rectangular image having the same shape as that of the input image, as indicated by the point of a white arrow in FIG. 12. In the image after the transformation, each dot included in the dot pattern is represented in a substantially perfect circular shape.).
Regarding claims 4 and 17, Akao discloses wherein the processor is configured to: identify a rectangular area of a maximum size corresponding to an aspect ratio of the input image in an area where the guide area and an area identified based on the third information are overlapped (para. 0182; a broken line frame F2, the correction vector generation unit 154 sets a maximum inscribed rectangle within the projection range), and perform keystone correction so that an image is projected on the identified rectangular area (para. 0182; calculates the correction vector for performing the geometric correction such that each dot is projected within the maximum inscribed rectangle).
Regarding claims 5 and 18, Akao discloses wherein the processor (123 and para. 0065) is configured to: based on the guide area being included in the area identified based on the third information (para. 0075; a guide, the user performs image capturing a plurality of times while changing the image capturing position), perform keystone correction to project an image onto a rectangular area of a maximum size corresponding to the aspect ratio of the input image in the guide area, and based on the area identified based on the third information being included in the guide area, perform keystone correction to project an image onto a rectangular area of a maximum size corresponding to the aspect ratio of the input image in the area identified based on the third information (para. 0182; a broken line frame F2, the correction vector generation unit 154 sets a maximum inscribed rectangle within the projection range), and perform keystone correction so that an image is projected on the identified rectangular area (para. 0182; calculates the correction vector for performing the geometric correction such that each dot is projected within the maximum inscribed rectangle).
Regarding claims 6 and 19, wherein: each of the plurality of markers is positioned in an area inwards by a preset ratio with reference to four vertexes of the test image (illustrated in fig. 12),
the processor (123 and para. 0065) is configured to obtain fourth information indicating positions of the four vertexes of the test image (para. 0156; corresponding point information for each of the captured images indicates the relationship between the camera corresponding points of each of the captured images captured at the corresponding position and the projector corresponding points) in the captured image based on the first information and the preset ratio (para. 0182; a broken line frame F2, the correction vector generation unit 154 sets a maximum inscribed rectangle within the projection range),
obtain the third information (illustrated in fig. 12; hatched dots in each corner of the solid dot image) of the captured image based on the fourth information and a transformation matrix (para. 0153; the corresponding point detection unit 151 transforms the projection image shown in the captured image on the basis of the homography matrix so as to obtain a substantially rectangular image having the same shape as that of the input image), and
the transformation matrix is obtained based on a mapping relation between the first information and the second information (para. 0151; the projection image of the dot pattern is included on the captured image in a distorted state according to the unevenness of the projection surface 3, the image capturing position, and the like. The fact that each dot included in the dot pattern is represented by a laterally long ellipse indicates that the projection image is distorted and para. 0152; In this case, the corresponding point detection unit 151 calculates, by a predetermined method, a homography matrix used to transform the coordinate system before the projective transformation into the coordinate system after the projective transformation.).
Regarding claims 7 and 20, Akao discloses wherein the processor (123 and para. 0065) is configured to correct the information about the guide area based on posture information of the external device (para. 0073; a display of the mobile terminal 1 displays a screen indicating the position of the projection surface and the positions at which image capturing has been finished. In the example of FIG. 4), and perform the keystone correction based on the third information and the corrected information about the guide area (para. 0077; mobile terminal 1 generates the correction data on the basis of the plurality of captured images captured in this manner, and transmits the correction data to the projector 2).
Regarding claim 9, Akao discloses a user terminal (mobile terminal 1 of fig. 1) comprising: a camera (para. 0082; a smartphone is illustrated as the mobile terminal 1, but another device such as a digital camera); a display (display 106 of fig. 5); a communication interface (communication unit 111 of fig. 5); and
a processor (123 and para. 0065) configured to: control the camera (camera 109 of fig. 5) to capture an image capturing a projection surface onto which a test image including a plurality of markers is projected (para. 0143; Coordinates of the dots on each captured image are detected by the pattern detection processing, and the relationship between the dots on each of the captured images and the dots on the dot pattern is determined), control the display to display a guide graphical user interface (GUI) for setting a guide area in the captured image (para. 0075; a guide, the user performs image capturing a plurality of times while changing the image capturing position), identify first information comprising coordinate information of an external projector device indicating a position of each of the plurality of markers (para. 0143; Coordinates of the dots on each captured image are detected by the pattern detection processing, and the relationship between the dots on each of the captured images and the dots on the dot pattern is determined), second information comprising coordinate information indicating a position of each of the plurality of markers based on the captured image, and information about the guide area (para. 0066; the user uses a camera function of the mobile terminal 1 to capture an image of the dot pattern projected by the projector 2 at a plurality of different image capturing positions. The captured images obtained by the image capturing show, for example, a range including the entire dot pattern projected as the projection image on the projection surface 3), obtain third information indicating positions of four vertexes of the test image in the captured image based on the first information and the second information (illustrated in fig. 12; hatched dots in each corner of the solid dot image),
obtain keystone correction information to project an image corresponding to the guide area based on the first information, the second information, and the information about the guide area (para. 0132; The correction data calculation processing #1 of FIG. 9 is a series of processing for calculating the correction data used for the geometric correction of the input image), and
transmit the obtained keystone correction to the external projector device through the communication interface (para. 0096; communication unit 122 receives the correction data transmitted from the mobile terminal 1 and outputs the correction data to the control unit 123).
Regarding claim 12, Akao discloses the processor (123 and para. 0065) is configured to based on a specific guide GUI being selected by a user input to adjust at least one of a size or a position of the guide GUI (para. 0075; a guide, the user performs image capturing a plurality of times while changing the image capturing position), obtain the information about the guide area corresponding to the selected guide GUI, and the information about the guide area comprises coordinate information about four vertex areas of the guide area in the captured image (illustrated in fig. 12; hatched dots in each corner of the solid dot image).
Regarding claim 13, Akao discloses wherein; the processor is configured to; control the display (illustrated in fig. 5) so that the guide GUI is overlapped with the captured image and displayed, and the guide GUI has a rectangular line shape (para. 0241; it is possible to present the information serving as the guide for the image capturing positions or the like on the basis of the estimated camera external parameter. In a case where an image of the projected dot pattern is captured again, information for guiding the user to an appropriate image capturing position may be presented).
Regarding claim 14, Akao discloses wherein the processor is configured to identify a recommended position to provide the guide GUI by analyzing the captured image and control the display to display the guide GUI and recommendation information on the recommended position (para. 0066; a camera function of the mobile terminal 1 to capture an image of the dot pattern projected by the projector 2 at a plurality of different image capturing positions. The captured images obtained by the image capturing show, for example, a range including the entire dot pattern projected as the projection image on the projection surface 3 and para. 0073; the position of the projection surface is displayed by use of an icon having a laterally-long rectangular shape, and each of the positions at which image capturing has been finished is displayed by use of an icon representing the appearance of the mobile terminal 1. The position of the projection surface and the positions at which image capturing has been finished are displayed on the basis of estimation results based on the captured images).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Akao (US PG Pub. 20220239876) as applied to claim 1 above, and further in view of Narukawa (JP2019169915A).
Regarding claim 8, Akao discloses a projection system (projector 2 of fig. 1) comprising an external mobile device (mobile terminal 1 of fig. 1) to assist in geometric image correction (para. 0077; mobile terminal 1 generates the correction data on the basis of the plurality of captured images captured in this manner, and transmits the correction data to the projector 2).
Akao fails to teach wherein each of the plurality of markers are in a pattern format in which a black area and a white area are configured by a preset ratio in each of a plurality of directions.
Narukawa discloses a projection system (projection device 10 of fig. 1) wherein
each of the plurality of markers (plurality of marker images 40 of figs. 6 and 7) are in a
pattern format in which a black area and a white area are configured by a preset ratio in
each of a plurality of directions (pg. 4 5th para.; a closed region of a black image and a white image of a binarized image).
It would have been obvious to one of ordinary skill in the art prior to the filing date
of the application to modify the markers of Akao with the black and white markers of
Narukawa in order to make it easier for the imaging device to distinguish the marker
from the background thereby providing correct information to the processing unit
reducing potential errors.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 9 and 15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANELL L OWENS whose telephone number is (571)270-5365. The examiner can normally be reached 9:00am-5:00pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Minh-Toan Ton can be reached at 571-272-2303. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANELL L OWENS/ Examiner, Art Unit 2882 26 May 2026
/CHRISTINA A RIDDLE/ Primary Examiner, Art Unit 2882
Prosecution Insights