DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 20-43 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 20-23, 32, 36, 40-43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 2020/0322586 A1) in view of Van Der Auwera et al. (US 10,652,553 B2) (hereinafter “Van”).
Consider claim 21, Lee teaches an apparatus comprising at least one processor ([0371] – [0373]) and at least one memory ([0371] – [0373]), said at least one memory stored with computer program code thereon ([0371] – [0373]), the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain an omnidirectional video media content ([0214] – [0217], [0221], Fig. 13, Fig. 14A-14B; [0251] – [0275]); determine a dominant axis for a viewport representation ([0251] – [0275]); and encode the omnidirectional video media content by aligning a yaw axis of the omnidirectional video media content with the dominant axis of the viewport representation ([0238], [0329] – [0346[).
However, Lee does not explicitly teach that the determination is based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content.
Van teaches the determination is based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content (col. 19, line 65 – col. 20, line 28; FIG. 4C is a diagram illustrating an example of representing a location of viewport 460 within the spherical space of spherical representation 410. In the example of FIG. 4C, the location of viewport 460 can be represented by a pitch angle 462 and a yaw angle 464. Both angles can be derived from a direction of view of the user based on the location of an ROI on the spherical scene. For example, a direction of view of the user positioned at spherical center 472 towards a viewport center 474 of the viewport 460 can be represented by a vector 470. Vector 470 may form a projection 476 on the x-z plane, and a projection 478 on the x-y plane. Pitch angle 462 can be formed between projection 476 and an axis 480 that is parallel with the y-axis. Yaw angle 464 can be formed between projection 478 and axis 480.
Both pitch angle 462 and yaw angle 464 can relate the location of viewport 460 with an orientation of the user's head and/or eyeballs. For example, pitch angle 462 can represent an elevation angle of vector 470, which can correspond to, for example, an angle of elevation of the user's head with respect to the x-z plane, a rotation of the user's eyeball with respect to the x-z plane, or any other movement of the user with respect to the x-z plane. Further, yaw angle 464 can represent a rotation angle of vector 470, which can correspond to, for example, an angle of rotation of the user's head, a rotation of the user's head with respect to the x-y plane, a rotation of the user's eyeball with respect to the x-y-plane, or any other movement of the user with respect to the x-z plane. By representing the location of viewport 460 based on pitch angle 462 and yaw angle 464, a location of region of interest (ROI) represented by viewport 460 can be determined efficiently based on the orientation of the user's head and/or eyeballs, which enables efficient rendering of the portion of spherical video content corresponding to the ROI.
In addition to the center 474 of viewport 460, other attributes of viewport 460 can also be represented based on the yaw angle 464 and the pitch angle 462. For example, referring to FIG. 4E, midpoints 482, 484, 486, and 488 can be midpoints between the edges of viewport 460.col. 22, line 8 – col. 24, line 22).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Consider claim 22, Lee teaches the obtained omnidirectional video media content comprises a first viewpoint representation, and the apparatus is further caused to: detect a need for a change of a viewport of the omnidirectional video media content from the first viewport representation into a second viewport representation ([0251] – [0275]); determine a dominant axis for the second viewport representation ([0251] – [0275]); and encode the omnidirectional video media content by aligning the yaw axis of the omnidirectional video media content with the dominant axis of the second viewport representation ([0238], [0324] – [0346]).
Consider claim 23, Lee teaches the apparatus is further caused to transmit the omnidirectional video media content to a second apparatus as viewport-dependent delivery ([0329] – [0346]).
Consider claim 20, claim 20 recites the method implemented by the apparatus recited in claim 21. Thus, it is rejected for the same reasons.
Consider claim 32, Lee teaches a method comprising: receiving an omnidirectional video media content encoded as viewport-dependent delivery into at least a first viewport representation (Fig. 1, Fig. 6, Fig. 13, [0065], [0131] – [0134], [0213] – [0214]); detecting a need for a change of a viewport into a second viewport representation ([0251] – [0275]); determining a dominant axis of the second viewport representation ([0251] – [0275]); and signaling the dominant axis of the second viewport representation to an apparatus encoding the omnidirectional video media content ([0066] – [0072], [0141] – [0142], [0176] – [0184]).
However, Lee does not explicitly teach that the determination is based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content.
Van teaches the determination is based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content (col. 19, line 65 – col. 20, line 28; FIG. 4C is a diagram illustrating an example of representing a location of viewport 460 within the spherical space of spherical representation 410. In the example of FIG. 4C, the location of viewport 460 can be represented by a pitch angle 462 and a yaw angle 464. Both angles can be derived from a direction of view of the user based on the location of an ROI on the spherical scene. For example, a direction of view of the user positioned at spherical center 472 towards a viewport center 474 of the viewport 460 can be represented by a vector 470. Vector 470 may form a projection 476 on the x-z plane, and a projection 478 on the x-y plane. Pitch angle 462 can be formed between projection 476 and an axis 480 that is parallel with the y-axis. Yaw angle 464 can be formed between projection 478 and axis 480.
Both pitch angle 462 and yaw angle 464 can relate the location of viewport 460 with an orientation of the user's head and/or eyeballs. For example, pitch angle 462 can represent an elevation angle of vector 470, which can correspond to, for example, an angle of elevation of the user's head with respect to the x-z plane, a rotation of the user's eyeball with respect to the x-z plane, or any other movement of the user with respect to the x-z plane. Further, yaw angle 464 can represent a rotation angle of vector 470, which can correspond to, for example, an angle of rotation of the user's head, a rotation of the user's head with respect to the x-y plane, a rotation of the user's eyeball with respect to the x-y-plane, or any other movement of the user with respect to the x-z plane. By representing the location of viewport 460 based on pitch angle 462 and yaw angle 464, a location of region of interest (ROI) represented by viewport 460 can be determined efficiently based on the orientation of the user's head and/or eyeballs, which enables efficient rendering of the portion of spherical video content corresponding to the ROI.
In addition to the center 474 of viewport 460, other attributes of viewport 460 can also be represented based on the yaw angle 464 and the pitch angle 462. For example, referring to FIG. 4E, midpoints 482, 484, 486, and 488 can be midpoints between the edges of viewport 460.col. 22, line 8 – col. 24, line 22).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Consider claim 36, Lee teaches an apparatus comprising at least one processor and at least one memory ([0371] – [0373]), said at least one memory stored with computer program code thereon ([0371] – [0373]), the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform the method recited in claim 32 (see rejection for claim 32).
Consider claim 40, Van teaches an equirectangular projection is utilized for projecting the omnidirectional video media content into a 2D rectilinear (col. Lines 20-24; col. 4, lines 53-54; col. 19, lines 15-58; col. 21, lines 3-14; col. 22, lines 8-56; col. 23, lines 1-10; col. 23, line 48 – col. 24, line 22; col. 24, line 65 – col. 25, line 8; col. 33, lines 24-40).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Consider claim 41, Van teaches an equirectangular projection is utilized for projecting the omnidirectional video media content into a 2D rectilinear (col. Lines 20-24; col. 4, lines 53-54; col. 19, lines 15-58; col. 21, lines 3-14; col. 22, lines 8-56; col. 23, lines 1-10; col. 23, line 48 – col. 24, line 22; col. 24, line 65 – col. 25, line 8; col. 33, lines 24-40).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Consider claim 42, Van teaches an equirectangular projection is utilized for projecting the omnidirectional video media content into a 2D rectilinear (col. Lines 20-24; col. 4, lines 53-54; col. 19, lines 15-58; col. 21, lines 3-14; col. 22, lines 8-56; col. 23, lines 1-10; col. 23, line 48 – col. 24, line 22; col. 24, line 65 – col. 25, line 8; col. 33, lines 24-40).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Consider claim 43, Van teaches an equirectangular projection is utilized for projecting the omnidirectional video media content into a 2D rectilinear (col. Lines 20-24; col. 4, lines 53-54; col. 19, lines 15-58; col. 21, lines 3-14; col. 22, lines 8-56; col. 23, lines 1-10; col. 23, line 48 – col. 24, line 22; col. 24, line 65 – col. 25, line 8; col. 33, lines 24-40).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining a dominant axis based upon an analysis of regions-of-interest or objects-of-interest in the omnidirectional video media content because such incorporation would facilitate the rendering of the 360-degree video. Col. 1, lines 61-67.
Claim(s) 24, 27-31, 33-35, 37-39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 2020/0322586 A1) in view of Van Der Auwera et al. (US 10,652,553 B2) (hereinafter “Van”) and Oyman (US 2020/0329088 A1).
Consider claim 24, Lee teaches all the limitations in claim 21 but does not explicitly teach the apparatus is further caused to obtain an indication of one or more dominant axes in connection with session negotiation.
Oyman teaches the apparatus is further caused to obtain an indication of one or more dominant axes in connection with session negotiation ([0160], [0235] – [0243], [0248] – [0257]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 27, Oyman teaches the apparatus is further caused to indicate support of alignment of the viewport in connection with the session negotiation ([0160], [0235] – [0243], [0248] – [0257]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 28, Oyman teaches the support of alignment of the viewport is indicated in a session description according to session description protocol (SDP), as an attribute for video, align with viewport change axis ([0160], [0235] – [0243], [0248] – [0257]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 29, Oyman teaches the indication of one or more dominant axes is configured to be obtained in a reply of the SDP session negotiation from the second apparatus ([0033], [0075] – [0111]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 30, Oyman teaches the indication of one or more dominant axes is configured to be obtained in an Real-time Transport Protocol (RTP) stream from the second apparatus ([0114] – [0120]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 31, Oyman teaches the support of alignment of the viewport is indicated in a session description according to a hypertext transfer protocol (HTTP) ([0160]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 33, Oyman teaches the dominant axis of one or more second viewport representation is configured to be signaled in a reply to a session description according to session description protocol (SDP) session negotiation received from the second apparatus ([0033], [0075] – [0111]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 34, Oyman teaches the dominant axis of one or more second viewport representation is configured to be signaled in an Real-time Transport Protocol (RTP) stream as a response to a session description according to Session Description Protocol (SDP) session negotiation received from the second apparatus ([0033], [0075] – [0111], [0114] – [0120]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 35, Oyman teaches the dominant axis of one or more second viewport representation is configured to be signaled in a reply to a hypertext transfer protocol (HTTP) session negotiation received from the second apparatus ([0160]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of obtaining an indication of one or more dominant axes in connection with session negotiation because such incorporation would ensure a reliable and interoperable service with a predictable media quality while allowing for flexibility in the service offerings. [0003].
Consider claim 37, claim 37 recites the apparatus that performs the method recited in claim 33.
Consider claim 38, claim 38 recites the apparatus that performs the method recited in claim 34.
Consider claim 39, claim 39 recites the apparatus that performs the method recited in claim 35.
Claim(s) 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 2020/0322586 A1) in view of Van Der Auwera et al. (US 10,652,553 B2) (hereinafter “Van”) and Stockhammer et al. (US 2019/0104326 A1).
Consider claim 25, Lee teaches all the limitations in claim 21 but does not explicitly teach the apparatus further caused to encode the omnidirectional video media content into the second viewport representation by including an IDR picture in the beginning of the encoded second viewport representation.
Stockhammer teaches the apparatus further caused to encode the omnidirectional video media content into the second viewport representation by including an IDR picture in the beginning of the encoded second viewport representation ([0291] – [0294]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of including an IDR picture in the beginning of the encoded second viewport representation because such incorporation would improve bandwidth utilization. [0302].
Claim(s) 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 2020/0322586 A1) in view of Van Der Auwera et al. (US 10,652,553 B2) (hereinafter “Van”) and He et al. (US 2019/0230142 A1).
Consider claim 26, Lee teaches all the limitations in claim 21 but does not explicitly teach the dominant axis is an axis of a most often viewed viewport, determined based on viewing orientation statistics of previous omnidirectional video media content.
He teaches the dominant axis is an axis of a most often viewed viewport, determined based on viewing orientation statistics of previous omnidirectional video media content ([0082], [0114] – [0118]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the known technique of determining the dominant axis based on viewing orientation statistics of previous omnidirectional video media content because such incorporation would improve the caching strategy. [0118].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAT CHI CHIO whose telephone number is (571)272-9563. The examiner can normally be reached Monday-Thursday 10am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JAMIE J ATALA can be reached at 571-272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAT C CHIO/Primary Examiner, Art Unit 2486